-- Logs begin at Thu 2022-06-30 09:18:49 UTC, end at Thu 2022-06-30 10:18:56 UTC. -- Jun 30 09:46:20.235847 dv-cinder-843309-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 09:46:20.247196 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 09:46:20 2022] *** Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: nodename: dv-cinder-843309-3-iscsi Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: machine: x86_64 Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: clock source: unix Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: pcre jit disabled Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: detected number of CPU cores: 4 Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: current working directory: / Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: detected binary path: /usr/bin/uwsgi-core Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: your processes number limit is 47680 Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: your memory page size is 4096 bytes Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: detected max file descriptor number: 2048 Jun 30 09:46:20.247770 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: lock engine: pthread robust mutexes Jun 30 09:46:20.248802 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: thunder lock: enabled Jun 30 09:46:20.248802 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 09:46:20.248802 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 09:46:20.299124 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: Python main interpreter initialized at 0x558bdcb039c0 Jun 30 09:46:20.299124 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: python threads support enabled Jun 30 09:46:20.299124 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: your server socket listen backlog is limited to 100 connections Jun 30 09:46:20.299124 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: your mercy for graceful operations on workers is 90 seconds Jun 30 09:46:20.299463 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 09:46:20.299525 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: *** Operational MODE: preforking *** Jun 30 09:46:20.299864 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: *** uWSGI is running in multiple interpreter mode *** Jun 30 09:46:20.299864 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: spawned uWSGI master process (pid: 80136) Jun 30 09:46:20.300045 dv-cinder-843309-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 09:46:20.300664 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: spawned uWSGI worker 1 (pid: 80149, cores: 1) Jun 30 09:46:20.301003 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: spawned uWSGI worker 2 (pid: 80150, cores: 1) Jun 30 09:46:20.301161 dv-cinder-843309-3-iscsi devstack@keystone.service[80136]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 09:46:25.691849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 09:46:25.691849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: from cryptography.utils import int_from_bytes Jun 30 09:46:25.692694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 09:46:25.692694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: from cryptography.utils import int_from_bytes Jun 30 09:46:25.692694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 09:46:25.692694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: from cryptography.utils import int_from_bytes Jun 30 09:46:25.693178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 09:46:25.693178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: from cryptography.utils import int_from_bytes Jun 30 09:46:25.740525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 09:46:25.740717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 09:46:25.740850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] command line args: [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 09:46:25.740938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 09:46:25.741071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 09:46:25.741148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 09:46:25.741286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] command line args: [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 09:46:25.741425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ================================================================================ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 09:46:25.741506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] admin_token = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.741590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 09:46:25.741751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ================================================================================ {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 09:46:25.741841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config_dir = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.741912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.741990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] admin_token = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] config_source = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] config_dir = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] debug = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] config_source = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] debug = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.742913 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.743929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_config_append = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_config_append = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_dir = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_dir = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_file = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.744961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.745944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.746872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.747814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] publish_errors = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] publish_errors = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.748961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.749893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] standard_threads = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750244 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] standard_threads = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_journal = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.750995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_json = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] use_journal = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_stderr = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] use_json = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] use_syslog = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] use_stderr = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.751947 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] use_syslog = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.752061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.752172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 09:46:25.752546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.752680 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.752795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.752907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.753993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.754929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.755982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.756975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.757937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.758960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.759924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.760948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.761931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.762931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.763040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.763149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.763563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.763682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.763799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.763937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.764953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.765944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.766961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.767085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.767207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.767553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.767667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.767779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.767935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.768997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.769994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.770913 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.771972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.772889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.773900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.774940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.775855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.776827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.external = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.external = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.777949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.password = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] auth.token = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.password = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] auth.token = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.778986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.779919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.780971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.781993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.782820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.783862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.784918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.785948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.786969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.787997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.788910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.789936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.790929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.791840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.792917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.793865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.794944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.795988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.796902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.797941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.798907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.799829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.800930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.801961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.802993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.803929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.804959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.805953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.806956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.807888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.808935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.809922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] role.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] role.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.810957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.811912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.812949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.813980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.814985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.815920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.816964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817880 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.817992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.818897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.819977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.820898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.821897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.822964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.connection = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.823931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.824944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.825970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80150) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 09:46:25.826654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 09:46:25.826654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 09:46:25.854738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.854973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.855183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.855269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.855441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.855669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.863493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.864050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.891488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.891683 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.891945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:25.892176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.036706 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.036898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.084697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.084921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.085265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.085501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.148912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.149102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.149183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.149247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.149331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.149461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.149628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.150094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.150192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.150261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.150418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.150543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.174627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.174781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.174916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.175960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.176037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.176175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.187120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.187472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.199907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.200995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.201168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.201251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.201315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.201388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80150) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.201510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.201510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 09:46:26.218793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.219012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 09:46:26.219783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.219783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 09:46:26.219783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.220144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.220144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 09:46:26.220377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.220377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.220560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.220685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.220768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 09:46:26.220768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.221068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.221323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.221576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.231323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.231595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.231858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.232163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.232422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.232703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.232973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.233253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.240359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.241503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.243646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.243941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.244214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.244495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.244770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.244893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.245105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.245197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.245331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.245491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.245614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.245749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.245875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.246957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.247227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.247496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.272476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.273931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.274124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.274347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.275599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.275850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.279614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.279903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.280299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.281167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.281444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.281840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.289432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.289432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.289678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.289678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.292350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.292632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.292906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.293182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.293451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.293725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.293941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.294065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.294201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.294281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.294501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.294782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.295040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.295312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.295578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.295845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.312711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.312985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.314493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.314769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.317553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.319439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.321180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.321567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.323295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.323593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.326613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.328752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.330204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.332383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.333841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.334137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.334418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.334710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.334997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.335271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.336066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.336364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.336651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.337482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.338155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.338426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.354648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.354926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.355192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.355449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.357576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.357862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.358132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.358516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.363291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.365033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.365306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.366268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.367999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.368309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.368594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.368866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.369144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.369421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.369703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.371495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.371770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.372077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.372353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.372628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.382982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.383260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.383533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.383941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.384198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.385812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.385936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.386074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.386226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.386533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.386816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.387110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.388783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.389088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.404033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.406967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.407257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.410265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.410572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.412316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.413612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.413947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.414228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.414507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.415388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.417065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.417352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.417638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.421821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.423406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.425004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.426598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.426742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.426945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.427287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.427566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.429848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.430145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.430534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.430867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.436940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.437228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.437503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.440042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.440337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.440614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.445371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80150) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.445645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.445920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.446187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.446518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.446901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.447180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.447450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.447748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.448045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.448323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.448568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.448657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 09:46:26.448839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.448924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.449102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80150) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.449196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.449485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.449758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.450139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.450426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.450692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.450986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.451247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.451520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.451784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.452132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.452337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 09:46:26.487079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558bdcb039c0 pid: 80150 (default app) Jun 30 09:46:26.490480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558bdcb039c0 pid: 80149 (default app) Jun 30 09:46:26.490604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bfd11-d6cf-4b8d-8dca-0e80dd35e15a None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:26.490772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bfd11-d6cf-4b8d-8dca-0e80dd35e15a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:26.491044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bfd11-d6cf-4b8d-8dca-0e80dd35e15a None None] PATH_INFO: `/v3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:26.495476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Jun 30 09:46:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:34.129264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:34.130187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:34.130187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:34.138031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:34.143966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:34.144414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:34.144506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:34.214519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: INFO dbcounter [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] Registered counter for database keystone Jun 30 09:46:34.239122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG oslo_db.sqlalchemy.engines [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80150) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 09:46:34.248558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writer thread running {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 09:46:34.427235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG passlib.handlers.bcrypt [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80150) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 09:46:34.427583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG passlib.handlers.bcrypt [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80150) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 09:46:34.483357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:34.485013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:34.618550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 09:46:34.854875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:34.924904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: INFO dbcounter [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] Registered counter for database keystone Jun 30 09:46:34.946547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG oslo_db.sqlalchemy.engines [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80149) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 09:46:34.958603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writer thread running {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 09:46:35.080455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcdb99-25ec-41e9-815d-8402ff63a23d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:35.085963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:35.086632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:35.087269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:35.087525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:35.096044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:35.147856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 09:46:35.148448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b02f3-d61f-4e51-8c3e-ea1372a574c1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:35.158754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Jun 30 09:46:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 09:46:35.165201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:35.180665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c45d2a-9158-428a-8f88-9036f7ede421 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:35.185807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:35.186961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:35.187255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:35.187517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] PATH_INFO: `/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:35.189142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] RBAC: Authorizing `identity:get_project(project_id=e04fd91b7cf34c948c016c3c7a5bd4df)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:35.242703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 09:46:35.243036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75db111-de2b-43dc-8b4a-df6d5983d2dd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:35.247757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Jun 30 09:46:35 2022] GET /identity/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:37.274180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaed62-23b9-499a-96b4-079a82ab5966 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:37.274710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaed62-23b9-499a-96b4-079a82ab5966 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:37.274815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaed62-23b9-499a-96b4-079a82ab5966 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:37.279173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:37.286105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:37.286519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:37.286819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:37.313137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:37.314669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:37.323383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 09:46:37.346595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afaed62-23b9-499a-96b4-079a82ab5966 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:37.361349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afaed62-23b9-499a-96b4-079a82ab5966 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:37.368841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb9b88-dcef-4cc7-a35b-024cf8fb4b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:37.369532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9b88-dcef-4cc7-a35b-024cf8fb4b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:37.369971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9b88-dcef-4cc7-a35b-024cf8fb4b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:37.370250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9b88-dcef-4cc7-a35b-024cf8fb4b5e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:37.380916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb9b88-dcef-4cc7-a35b-024cf8fb4b5e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:37.383829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb9b88-dcef-4cc7-a35b-024cf8fb4b5e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:37.398810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Jun 30 09:46:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 09:46:37.407116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:37.422052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4055-0953-486b-aa71-215c90c41849 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:37.431534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f9a6a-f4cf-4193-aa92-6aafe535356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:37.432587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f9a6a-f4cf-4193-aa92-6aafe535356f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:37.432867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f9a6a-f4cf-4193-aa92-6aafe535356f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:37.433143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f9a6a-f4cf-4193-aa92-6aafe535356f admin admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:37.434784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f9a6a-f4cf-4193-aa92-6aafe535356f admin admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:37.436670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f9a6a-f4cf-4193-aa92-6aafe535356f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:37.438272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Jun 30 09:46:37 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:39.507303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9fff8-c26b-48f7-97d8-38e27c16ef11 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.507854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9fff8-c26b-48f7-97d8-38e27c16ef11 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.507854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9fff8-c26b-48f7-97d8-38e27c16ef11 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.509200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:39.514716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.515131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.515301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.532924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:39.534123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:39.557070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c3baa-2d40-4aa6-a906-b72cce7f9a9c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.557274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c3baa-2d40-4aa6-a906-b72cce7f9a9c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.557545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c3baa-2d40-4aa6-a906-b72cce7f9a9c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.558906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:39.564026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.564311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.564583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.604992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG passlib.handlers.bcrypt [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80149) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 09:46:39.605457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG passlib.handlers.bcrypt [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80149) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 09:46:39.616533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:39.640583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:39.650684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:39.652308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:39.665394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:39.747024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9eaae5-1b59-4c24-8f83-8ed0a4c735b1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:39.754334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:39.776996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:39.782193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:39.782997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.783278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.783554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.795756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:39.806903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:39.807812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:46:39.809108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:46:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:39.814244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:39.879258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4242db-45b7-498c-9338-68612a7be76d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:39.881548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:39.894246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:39.913480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54788fdb-abef-4928-8751-7f01e9e2d556 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:39.918438 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:39.919350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.919638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.919957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.948095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:39.948765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:39.949046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:39.949322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:39.950078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:39.950385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:39.951803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:39.963892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:39.969209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:39.988618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:39.990161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:40.069663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c82cfb-15c4-4a82-9cf7-6b70dffd3cda None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:40.093522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:46:40.099066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:40.102363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:40.106268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:40.108148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:40.108445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:40.108902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:40.110275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:40.111769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:40.113475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:39 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:40.119060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:40.133986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:40.196357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc86233-f496-4977-891b-e9597f8cf16b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:40.217985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138fbb8-6745-4831-ab54-c4b9fa8965d6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:40.229889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:40.230690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:40.230975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:40.231285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:40.251242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:40.252098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:40.252756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:40.252907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:40.253192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:40.253560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:40.254778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:40.256256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:40.257872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:40.263685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:46:40.278295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:40.279143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:40.280341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:46:40.280341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:46:40.280341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:46:40.280341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:40.280341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:40.285201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:40.343390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:40.346838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:40.365771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196bb7-7cb4-4a29-9b98-e084cee4f462 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:40.388224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e59e6-5506-4e2b-98fd-31c81845a553 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:40.401158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:40.401834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:40.402112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:40.402383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:40.403228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.domain.id=default, user.id=7c92e7defa704085b3b2760be29aa346) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:40.403539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:40.405040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee431b-739d-40b0-aa00-e2c51d7e02d3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:40.411841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Jun 30 09:46:40 2022] GET /identity/v3/role_assignments?user.id=7c92e7defa704085b3b2760be29aa346&scope.domain.id=default&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:40.421596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:40.422276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:40.422558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:40.422831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:40.423579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:40.423915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:40.425373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d6ab-70e3-42ea-8ce8-9bb10e78afb9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:40.445825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:42.736803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fca56b-9f39-4f07-a0f9-9300765dba76 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:42.737344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fca56b-9f39-4f07-a0f9-9300765dba76 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:42.737344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fca56b-9f39-4f07-a0f9-9300765dba76 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:42.738718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:42.739269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4e936-105f-432e-8902-cd5b65c68861 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:42.740041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4e936-105f-432e-8902-cd5b65c68861 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:42.740504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4e936-105f-432e-8902-cd5b65c68861 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:42.742058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 11/23] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:46:42.743647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:42.744071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:42.744386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:42.747973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351f7df-8182-4147-901e-b0939a31a8a4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:42.748473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351f7df-8182-4147-901e-b0939a31a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:42.748965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351f7df-8182-4147-901e-b0939a31a8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:42.772899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:42.774106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:42.774450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1351f7df-8182-4147-901e-b0939a31a8a4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:42.775845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1351f7df-8182-4147-901e-b0939a31a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:42.881057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 12/24] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:46:42.882999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb5fef-900e-4f4a-a637-712edeb08ad8 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:42.883478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb5fef-900e-4f4a-a637-712edeb08ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:42.883975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb5fef-900e-4f4a-a637-712edeb08ad8 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:42.885588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 13/25] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:42.890144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 13/26] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:42.891124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:42.891619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:42.892113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:42.908598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:42.910067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:42.921375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:42.957273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:42.992675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 14/27] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:46:42.999505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.042809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:43.069963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741be5e6-282b-4b2f-bb28-40e3c407cca7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.106467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.108812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.109286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.110440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.129411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:43.137384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:43.138197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:43.138965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:46:43.141749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 14/28] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:46:43.142747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.218255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5147c-797d-4487-b392-9cf141818338 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.254423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43dd7b-48a2-4e7f-8e01-f83ea757557e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.274732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.275632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.288326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.288729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.291935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.293497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.294182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.294650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.295612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.297314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:43.300394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:43.301773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.303681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:46:43.329792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:43.330665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:43.331459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:46:43.331459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:46:43.331459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:46:43.331459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:43.333430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 15/29] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:43.334419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.369552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:43.447681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a9d14-44c2-4347-9ff2-56e74cf11111 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.483042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.484083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.484741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.485209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.486163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:43.486660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.488292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:43.497138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 15/30] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 09:46:43 2022] POST /identity/v3/roles => generated 219 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:43.504095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.507144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 16/31] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:43.514175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.620732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:43.630771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50d96-b698-42b3-bf06-83a2233e2123 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.660451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ba463-675c-49cd-a6aa-a6518166200b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.669409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4f98b-68ce-42b1-b606-e829d55938b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.670602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4f98b-68ce-42b1-b606-e829d55938b7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.671100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4f98b-68ce-42b1-b606-e829d55938b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.671587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4f98b-68ce-42b1-b606-e829d55938b7 None admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.673663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4f98b-68ce-42b1-b606-e829d55938b7 None admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.675429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4f98b-68ce-42b1-b606-e829d55938b7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:43.677509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 17/32] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:43 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:43.679125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5822b-7178-4b41-8ba1-00093d76b62c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.679632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5822b-7178-4b41-8ba1-00093d76b62c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.680140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5822b-7178-4b41-8ba1-00093d76b62c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.681711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 18/33] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:43.683741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5822b-7178-4b41-8ba1-00093d76b62c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.707864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.708629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.708921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.709202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.710021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:43.710373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.712604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa786e-5b3e-41fc-aed5-b87914c2037a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:43.734311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 16/34] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:43.735968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.736352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.736679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.757519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:43.758894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:43.801575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5822b-7178-4b41-8ba1-00093d76b62c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.819685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:43.840192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.841667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.842219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.842689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.843198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 17/35] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:43.844160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.844556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.846241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:43.848099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 19/36] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:43.854736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:43.886609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:43.954364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab93fa-5ea4-4477-8e26-cbf993c90c49 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.965787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f12ed-4838-41a1-8a8f-220aced969e9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:43.990145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:43.990914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:43.991089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:43.991372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:43.992122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:43.993604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:44.004124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cd72d-ed5a-4755-8da5-5ae46ee7417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:44.005437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cd72d-ed5a-4755-8da5-5ae46ee7417d None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.005973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cd72d-ed5a-4755-8da5-5ae46ee7417d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.006437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cd72d-ed5a-4755-8da5-5ae46ee7417d None admin] PATH_INFO: `/v3/domains/default/users/7c92e7defa704085b3b2760be29aa346/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.008863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cd72d-ed5a-4755-8da5-5ae46ee7417d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7c92e7defa704085b3b2760be29aa346, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:44.010743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cd72d-ed5a-4755-8da5-5ae46ee7417d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:44.038274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:46:43 2022] POST /identity/v3/projects => generated 301 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:44.039052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:44.083595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:44.147166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 20/38] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Jun 30 09:46:43 2022] PUT /identity/v3/domains/default/users/7c92e7defa704085b3b2760be29aa346/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 09:46:44.324619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929b3e2-c143-4f21-9df6-726006cff7f7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:44.378158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39a9be-4d32-4561-b5e8-db02b3999fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:44.378849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39a9be-4d32-4561-b5e8-db02b3999fa9 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.379138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39a9be-4d32-4561-b5e8-db02b3999fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.379418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39a9be-4d32-4561-b5e8-db02b3999fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.380244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39a9be-4d32-4561-b5e8-db02b3999fa9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:44.381665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39a9be-4d32-4561-b5e8-db02b3999fa9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:44.404091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 19/39] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 09:46:44 2022] POST /identity/v3/roles => generated 225 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:44.490781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1770f6-09c5-4420-848c-f8e305104a2c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.491139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1770f6-09c5-4420-848c-f8e305104a2c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.491346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1770f6-09c5-4420-848c-f8e305104a2c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.495967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 21/40] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:44.501923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.502216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.502501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.520746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:44.521977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:44.530486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22591937-348e-4ed3-93aa-955b9c16b050 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.530771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22591937-348e-4ed3-93aa-955b9c16b050 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.531051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22591937-348e-4ed3-93aa-955b9c16b050 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.533271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 22/41] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:44.538876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.539164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.539448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.557655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:44.559123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:44.603437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 20/42] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:44.632960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:44.638071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 23/43] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:44.662074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:44.662297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:44.681160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:44.741751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d39d6a-f6d0-45ce-b936-87eb83a45cff None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:44.773879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab785b-c25b-4b34-893c-9f427360781e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:44.778148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:44.778958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.779241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.779516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.780733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:44.782226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:44.784087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 21/44] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:44.789761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:44.811567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d5c32-185e-4790-a620-fd74dd1a2472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:44.812236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d5c32-185e-4790-a620-fd74dd1a2472 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.812493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d5c32-185e-4790-a620-fd74dd1a2472 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.812769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d5c32-185e-4790-a620-fd74dd1a2472 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.813525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d5c32-185e-4790-a620-fd74dd1a2472 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:44.815080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d5c32-185e-4790-a620-fd74dd1a2472 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:44.833021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 24/45] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 09:46:44 2022] POST /identity/v3/roles => generated 223 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:46:44.906360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:44.915148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff8f12-7912-4ced-850b-67fa0764d9e2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:44.952864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfb010-63a4-426a-b48b-5cfbb24303aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:44.953556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfb010-63a4-426a-b48b-5cfbb24303aa None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:44.953842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfb010-63a4-426a-b48b-5cfbb24303aa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:44.954121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfb010-63a4-426a-b48b-5cfbb24303aa None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:44.954871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfb010-63a4-426a-b48b-5cfbb24303aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:44.956429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfb010-63a4-426a-b48b-5cfbb24303aa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:44.992266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 22/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:46:44 2022] POST /identity/v3/projects => generated 312 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:46:46.306996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62c0d-19fe-4dda-a53c-18a72b87eae0 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:46.307459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62c0d-19fe-4dda-a53c-18a72b87eae0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:46.307529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62c0d-19fe-4dda-a53c-18a72b87eae0 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:46.309133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 25/47] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:46.314345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:46.314748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:46.314876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:46.332919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:46.334161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:46.402754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 23/48] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:46.425942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62c0d-19fe-4dda-a53c-18a72b87eae0 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:46.507365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:46.532310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62c0d-19fe-4dda-a53c-18a72b87eae0 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:46.565873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:46.566603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:46.566881 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:46.567153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:46.579811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:46.588824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:46.589885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:46.590762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 26/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:46.596324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:46.677652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:46.703126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c13ea-f0c5-45b7-9cbc-bbe14288efab None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:46.740303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:46.740981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:46.741268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:46.741547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:46.742318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:46.742627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:46.744608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:46.752973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 24/50] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:46.758710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:46.866805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44349133-8623-49a2-93b9-8ff3cb78b091 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:46.901160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa21fbc-38e8-4637-bd12-b1e2bc7a973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:46.902038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21fbc-38e8-4637-bd12-b1e2bc7a973c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:46.902320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21fbc-38e8-4637-bd12-b1e2bc7a973c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:46.902591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21fbc-38e8-4637-bd12-b1e2bc7a973c None admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:46.903836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa21fbc-38e8-4637-bd12-b1e2bc7a973c None admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:46.905326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa21fbc-38e8-4637-bd12-b1e2bc7a973c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:46.906955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 27/51] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:46 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:46.912576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:47.007503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ff482-2858-43dd-a56b-b350ad0b1649 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:47.007753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ff482-2858-43dd-a56b-b350ad0b1649 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:47.008059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ff482-2858-43dd-a56b-b350ad0b1649 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:47.009580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 28/52] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:46:47.014519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c180-c4ee-4e34-a2db-ed36c16ed1ec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:47.015037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:47.015290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:47.015591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:47.022255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:47.035860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:47.037241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:47.059466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:47.060318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:47.060609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:47.060885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:47.062156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:47.063637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:47.065327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 25/53] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:47.071031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:47.080398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:47.109135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 29/54] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:47.132542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:47.175573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1c15f-cf7e-47e8-ab2b-dece871f52e2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:47.210299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:47.210984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:47.211257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:47.211532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:47.212829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.domain.id=default, user.id=7c92e7defa704085b3b2760be29aa346) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:47.213276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:47.215057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:47.220968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 26/55] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Jun 30 09:46:47 2022] GET /identity/v3/role_assignments?user.id=7c92e7defa704085b3b2760be29aa346&scope.domain.id=default&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:47.231117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43975-1f0c-4d0a-b704-5d659bb004be None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:47.382119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32df85-3db7-4594-86f5-71a3ec80613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:47.382993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32df85-3db7-4594-86f5-71a3ec80613d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:47.383217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32df85-3db7-4594-86f5-71a3ec80613d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:47.383471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32df85-3db7-4594-86f5-71a3ec80613d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:47.384775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32df85-3db7-4594-86f5-71a3ec80613d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:47.386405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32df85-3db7-4594-86f5-71a3ec80613d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:47.388539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 30/56] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:47.393913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:47.467429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:47.505648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2675a-6c1f-4913-808c-24152740478f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:47.542857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e171f5a-3a3f-43a7-a072-ed38e8e0a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:47.543517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e171f5a-3a3f-43a7-a072-ed38e8e0a9a9 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:47.543807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e171f5a-3a3f-43a7-a072-ed38e8e0a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:47.544126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e171f5a-3a3f-43a7-a072-ed38e8e0a9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:47.544857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e171f5a-3a3f-43a7-a072-ed38e8e0a9a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:47.546348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e171f5a-3a3f-43a7-a072-ed38e8e0a9a9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:47.574768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 27/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Jun 30 09:46:47 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:49.602186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c15ef-520e-4f52-9b26-7ea260004109 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:49.603029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c15ef-520e-4f52-9b26-7ea260004109 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:49.603029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c15ef-520e-4f52-9b26-7ea260004109 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:49.605604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 31/58] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:49.611851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:49.612528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:49.612822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:49.639481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:49.640710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:49.710824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 28/59] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:49.735811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c15ef-520e-4f52-9b26-7ea260004109 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:49.739451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:49.848154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c15ef-520e-4f52-9b26-7ea260004109 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:49.882561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793dd7-d7a5-4867-9c51-e0f5fa128909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:49.883272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793dd7-d7a5-4867-9c51-e0f5fa128909 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:49.883553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793dd7-d7a5-4867-9c51-e0f5fa128909 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:49.883826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793dd7-d7a5-4867-9c51-e0f5fa128909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:49.885236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793dd7-d7a5-4867-9c51-e0f5fa128909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:49.886684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793dd7-d7a5-4867-9c51-e0f5fa128909 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:49.888327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 32/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:49.897062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:49.952580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:50.001832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816afe6a-ad90-4826-8b86-1ea242e1dc24 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:50.041205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1872e-d4f6-47cf-ac00-64d574ce6012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:50.041924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1872e-d4f6-47cf-ac00-64d574ce6012 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:50.042186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1872e-d4f6-47cf-ac00-64d574ce6012 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:50.042464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1872e-d4f6-47cf-ac00-64d574ce6012 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:50.043187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1872e-d4f6-47cf-ac00-64d574ce6012 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:50.044704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1872e-d4f6-47cf-ac00-64d574ce6012 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:50.077479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 29/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:46:49 2022] POST /identity/v3/users => generated 275 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:52.948969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ce309-509a-4618-8267-c2cace36140a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:52.949486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ce309-509a-4618-8267-c2cace36140a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:52.949592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ce309-509a-4618-8267-c2cace36140a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:52.951058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 33/62] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:52.956315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:52.956724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:52.956896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:52.984827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:52.986013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:53.073659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 30/63] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:46:53.125456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ce309-509a-4618-8267-c2cace36140a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.203808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:53.249541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ce309-509a-4618-8267-c2cace36140a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:53.295900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59165cf5-4a8f-42e9-a1da-2b44c2b3ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:53.295900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165cf5-4a8f-42e9-a1da-2b44c2b3ff6b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.295900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165cf5-4a8f-42e9-a1da-2b44c2b3ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.295900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165cf5-4a8f-42e9-a1da-2b44c2b3ff6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.305403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59165cf5-4a8f-42e9-a1da-2b44c2b3ff6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-59165cf5-4a8f-42e9-a1da-2b44c2b3ff6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:53.313325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:53.314358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:53.315290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:53.317312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 34/64] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:53.323425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.377165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:53.379105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea5a9-ce76-4b51-a541-ac3008a9aabb None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.379587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea5a9-ce76-4b51-a541-ac3008a9aabb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.380092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea5a9-ce76-4b51-a541-ac3008a9aabb None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.381657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 35/65] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:53.388756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0784d-12f9-4405-9a94-f585b1131d24 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.389039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0784d-12f9-4405-9a94-f585b1131d24 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.390505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0784d-12f9-4405-9a94-f585b1131d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.412074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-26d0784d-12f9-4405-9a94-f585b1131d24 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:53.413302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-26d0784d-12f9-4405-9a94-f585b1131d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:53.483651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297d5f-4cdd-4e13-902f-a5bbaf88cb2b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:53.521728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:53.522449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 36/66] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:53.522986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.523459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.523952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.524924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:53.525504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:53.527111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:53.528615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a187c-c058-4e8a-8f34-20b7b1b8082c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.529023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a187c-c058-4e8a-8f34-20b7b1b8082c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.529189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a187c-c058-4e8a-8f34-20b7b1b8082c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.530583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 37/67] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:53.537092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.537375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.537646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.547260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 31/68] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:53.556975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.566816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:53.568065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:53.627540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:53.648261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 38/69] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:53.649510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.676916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:53.683275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c79d5-3c56-486c-b398-57936a017021 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:53.718013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:53.719289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.719780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.720279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.734157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:53.736011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:53.737936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 32/70] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:53.738897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.779568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f2db5-3d43-407e-8795-c5f14bd03e28 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:53.819471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:53.820310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.820603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.820874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.831652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:53.839583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4247-c478-404a-9e19-2a98992352be None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:53.839715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:53.840555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:53.841750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 39/71] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:53.842580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.879858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4a0b3-8d9d-4761-82c3-c1fdf276e367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:53.880691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a0b3-8d9d-4761-82c3-c1fdf276e367 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.880970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a0b3-8d9d-4761-82c3-c1fdf276e367 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.881245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4a0b3-8d9d-4761-82c3-c1fdf276e367 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.893264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4a0b3-8d9d-4761-82c3-c1fdf276e367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:53.894147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d4b4a0b3-8d9d-4761-82c3-c1fdf276e367 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:53.901625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:53.902653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:46:53.903528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 33/72] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:53.904320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c13d0-a5d9-4791-86e9-5b5c7c60648b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.904555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c13d0-a5d9-4791-86e9-5b5c7c60648b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.904834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c13d0-a5d9-4791-86e9-5b5c7c60648b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:53.906088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 34/73] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:53.906776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c13d0-a5d9-4791-86e9-5b5c7c60648b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:53.926948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:53.962200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65df1e1-4f76-469f-a440-860b88a1b8ac None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:53.998835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce6637-21b0-4000-b595-bd73746d1256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:53.999408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6637-21b0-4000-b595-bd73746d1256 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:53.999690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6637-21b0-4000-b595-bd73746d1256 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:53.999991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6637-21b0-4000-b595-bd73746d1256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.011971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c13d0-a5d9-4791-86e9-5b5c7c60648b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.045807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.046485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.046768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.047043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.047801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:54.048138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.049536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.060057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 35/74] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.060057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.069343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce6637-21b0-4000-b595-bd73746d1256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.070831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce6637-21b0-4000-b595-bd73746d1256 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.118288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.139194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.158738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e564d-2c7a-420f-8b1a-ebb3a102e45c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.171226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 40/75] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:46:54.172531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.172824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.173103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.189277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:54.190636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:54.196496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.197255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.197538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.197826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.198583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:54.198895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.200311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d8af3-8b63-442c-8f2f-c51fbf5a5cf6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.208320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 36/76] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.209526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9875526-7ebb-460d-8365-b76f1af231be None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.209816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9875526-7ebb-460d-8365-b76f1af231be None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.210097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9875526-7ebb-460d-8365-b76f1af231be None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.211353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 37/77] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:54.212995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9875526-7ebb-460d-8365-b76f1af231be None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.259576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 41/78] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:54.260462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.319446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9875526-7ebb-460d-8365-b76f1af231be None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.340742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.353813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.354685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.354964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.355246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.356450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.357967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.359618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 38/79] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.360403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.362099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e6899-81b3-4494-a912-09307a0385a7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.371844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.395843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592e9a4-6f9c-4276-8a1f-7d0b12675b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.396748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e9a4-6f9c-4276-8a1f-7d0b12675b1f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.397034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e9a4-6f9c-4276-8a1f-7d0b12675b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.397312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e9a4-6f9c-4276-8a1f-7d0b12675b1f None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.398502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592e9a4-6f9c-4276-8a1f-7d0b12675b1f None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.400028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592e9a4-6f9c-4276-8a1f-7d0b12675b1f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.402003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 42/80] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.406758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.406758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.406758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.420299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:54.421469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:54.464241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff3fc-4491-43f2-bf03-0efc919f8836 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.487237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 43/81] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:54.488277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.497653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.498540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.498819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.499140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.500542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.502054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.503927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 39/82] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.508842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.564761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.585588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.587181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9661e29-fcd9-42e8-b287-b2d6b37f1761 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.607527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efc388-13ea-45f0-a1e8-76ecf8da3409 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.620286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.621072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.621350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.621622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.631566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:54.639329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:54.640201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:54.640967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:54.641545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.641633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 44/83] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:54.641633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.642306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.642306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.642550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.712348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.713897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.739312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040cbde-d957-434e-b277-acf354fbac45 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.773784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.774440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.774725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.775036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.775901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=0e717e6887484c4c914aefb3e73d3210) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:54.776227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.777700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.785451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 45/84] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/role_assignments?user.id=0e717e6887484c4c914aefb3e73d3210&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.786226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.789586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.799575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:54.815705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 40/85] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:46:54.816535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:54.929729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1479-f974-4566-b007-a7c3da7b18e4 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.931822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8e54e-9154-4186-8583-c429b079dd70 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:54.966552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.967199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.967491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.967765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.975883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:54.976904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:54.977320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:54.977746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:54.979255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:54.980893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:54.983010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 41/86] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:54.984078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.033654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.044352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.045841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.047702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.093476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f782e-dc06-4792-9457-cb86470025f6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.130984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.131668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.131984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.132265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.133038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:55.133346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.134803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.143245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 42/87] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:55.144089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.157851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 46/88] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:46:55.158670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.253358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3abc0-07f8-4b5e-be78-e06bf372bad6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.263438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.268756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f29bb6-9729-4d73-9da4-72e3fe9f5046 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.270325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.298103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.299136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.299410 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.299682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.300929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.302496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.304417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 43/89] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:55.305014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.316527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.317218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.318004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.318297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.319031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.321410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.354906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 47/90] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:46:55 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:55.356100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.449396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e00e42-c502-4d8b-b7a0-3a23de1940a6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.476834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8252fb-badb-4bfc-b299-100934de08f2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.503218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.505206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.505960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.506467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.508767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.511594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.514455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 44/91] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:55.515653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.516012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.517031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.517461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.517833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.519247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.521455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.523345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 48/92] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:55.525053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.561017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.575122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.639144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6faf39-daf7-46b4-9fdd-dd3308215ea9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.642345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b02ec-5f4c-43b9-a4f1-9b9640f35995 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.682635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.683309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.683591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.683938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.684767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=7c92e7defa704085b3b2760be29aa346) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:55.685076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.686537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.688993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.689653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.689931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.690202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.696152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 45/93] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/role_assignments?user.id=7c92e7defa704085b3b2760be29aa346&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:55.697103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:55.776319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.777846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.808973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.830373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5632a-68dd-4ce8-85ff-274b80758a06 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:55.841690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:55.871249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:55.872429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:55.872996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:55.873468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:55.874564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] RBAC: Adding query filter params (role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=0e717e6887484c4c914aefb3e73d3210) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:55.875094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:55.876903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:55.883511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 46/94] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/role_assignments?user.id=0e717e6887484c4c914aefb3e73d3210&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=805a0b1c9f554aff8a16769124f939eb => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:55.889371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 49/95] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:46:55.895224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:56.025327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c74a0-ddb1-4ede-a38c-2db62b1e4f90 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:56.064560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8289d-b94b-4848-85f3-7160cd8892b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:56.065456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8289d-b94b-4848-85f3-7160cd8892b4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:56.065743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8289d-b94b-4848-85f3-7160cd8892b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:56.066020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8289d-b94b-4848-85f3-7160cd8892b4 None admin] PATH_INFO: `/v3/projects/4bb4a8a066894ff7974f0825bc04728d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:56.067258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8289d-b94b-4848-85f3-7160cd8892b4 None admin] RBAC: Authorizing `identity:get_project(project_id=4bb4a8a066894ff7974f0825bc04728d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:56.068809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8289d-b94b-4848-85f3-7160cd8892b4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:56.070795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:46:55 2022] GET /identity/v3/projects/4bb4a8a066894ff7974f0825bc04728d => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:56.077791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:56.187637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6cbb1-fea8-411b-9f26-8f416bc47979 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:56.221230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:56.233469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:56.234146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:56.234427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:56.234746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:56.235606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=4bb4a8a066894ff7974f0825bc04728d, user.id=0e717e6887484c4c914aefb3e73d3210) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:56.235939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:56.237401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a10f43-242f-44c4-a29b-04b6457c9b4e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:56.245114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:46:56 2022] GET /identity/v3/role_assignments?user.id=0e717e6887484c4c914aefb3e73d3210&scope.project.id=4bb4a8a066894ff7974f0825bc04728d&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:57.207271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1564-b0da-4018-8bd0-9603e2183137 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:57.207830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1564-b0da-4018-8bd0-9603e2183137 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:57.207830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1564-b0da-4018-8bd0-9603e2183137 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:57.209257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:57.215188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:57.216034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:57.216514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:57.240306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:57.241846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:57.325252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 51/99] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:57.359774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1564-b0da-4018-8bd0-9603e2183137 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:57.421899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:57.485604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1564-b0da-4018-8bd0-9603e2183137 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:57.527624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:57.528541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:57.528830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:57.529110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:57.540174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:57.552639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:57.553574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:46:57.554315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 49/100] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:57.561158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:57.703602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6760-e281-4e6b-9955-89a6fdb02d38 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:57.724717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:57.739140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:57.739800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:57.740126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:57.740405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:57.741173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:57.741484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:57.742902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:57.751404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 52/101] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:46:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:57.757005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:57.864569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:57.867644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9629f-b17c-47b9-8b4b-c562c3fe6317 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:57.903328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:57.904253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:57.904541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:57.904818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:57.906025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:57.907483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:57.909317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 50/102] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:46:57 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:46:57.914768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:58.044723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b490a-7367-4d05-af03-174540d29395 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:58.089243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80881bfd-af8f-495d-84fc-60d7c2faf62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:58.090124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881bfd-af8f-495d-84fc-60d7c2faf62f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:58.090407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881bfd-af8f-495d-84fc-60d7c2faf62f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:58.090689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881bfd-af8f-495d-84fc-60d7c2faf62f None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:58.091901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80881bfd-af8f-495d-84fc-60d7c2faf62f None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:58.093388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80881bfd-af8f-495d-84fc-60d7c2faf62f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:58.095311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 53/103] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:46:57 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:58.100942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:58.203369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddf17a-665c-4df9-9638-c59ea7e2e908 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:58.238856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb3ba1-4690-4e52-977d-a1b0d8d40dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:58.239813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb3ba1-4690-4e52-977d-a1b0d8d40dfa None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:58.240132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb3ba1-4690-4e52-977d-a1b0d8d40dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:58.240461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb3ba1-4690-4e52-977d-a1b0d8d40dfa None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f/users/0e717e6887484c4c914aefb3e73d3210/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:58.242450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb3ba1-4690-4e52-977d-a1b0d8d40dfa None admin] RBAC: Authorizing `identity:create_grant(project_id=f525943eff304b3786a90a844c39a37f, user_id=0e717e6887484c4c914aefb3e73d3210, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:58.244041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb3ba1-4690-4e52-977d-a1b0d8d40dfa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:58.267496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 51/104] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:46:58 2022] PUT /identity/v3/projects/f525943eff304b3786a90a844c39a37f/users/0e717e6887484c4c914aefb3e73d3210/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:46:58.924750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b39d2c-6a17-44c1-a114-b0ca5f3e7a8f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:58.925732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b39d2c-6a17-44c1-a114-b0ca5f3e7a8f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:58.926217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b39d2c-6a17-44c1-a114-b0ca5f3e7a8f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:58.927811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 54/105] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:58.933643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7be07-f5f4-435c-bc7a-12abe0609ec1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:58.934218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7be07-f5f4-435c-bc7a-12abe0609ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:58.934700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7be07-f5f4-435c-bc7a-12abe0609ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:58.956993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-97d7be07-f5f4-435c-bc7a-12abe0609ec1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:58.958503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-97d7be07-f5f4-435c-bc7a-12abe0609ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:58.968239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32324655-da98-4506-a4da-22ab34465f78 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:58.968843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32324655-da98-4506-a4da-22ab34465f78 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:58.969311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32324655-da98-4506-a4da-22ab34465f78 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:58.970889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 55/106] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:58.975780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40587c52-9fb1-46dd-b8a3-46c519614de0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:58.976321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40587c52-9fb1-46dd-b8a3-46c519614de0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:58.976807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40587c52-9fb1-46dd-b8a3-46c519614de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.004860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-40587c52-9fb1-46dd-b8a3-46c519614de0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:59.006427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-40587c52-9fb1-46dd-b8a3-46c519614de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:59.012027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.072465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 52/107] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:59.074128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4de90-b8e0-4796-9146-e55dcdd22e25 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.074558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4de90-b8e0-4796-9146-e55dcdd22e25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.074985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4de90-b8e0-4796-9146-e55dcdd22e25 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.076425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 53/108] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:59.077608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d31d4-608d-4c40-935b-5a049b0673d2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.078099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d31d4-608d-4c40-935b-5a049b0673d2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.078584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d31d4-608d-4c40-935b-5a049b0673d2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.080656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 54/109] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:59.081749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22072e4-ef69-4ef9-bfdb-4b997abef16c None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.082037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22072e4-ef69-4ef9-bfdb-4b997abef16c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.082317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22072e4-ef69-4ef9-bfdb-4b997abef16c None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.083702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 55/110] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:46:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:46:59.089108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87d01-5360-417b-bb8f-88e9714f3cff None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.089395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87d01-5360-417b-bb8f-88e9714f3cff None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.089826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87d01-5360-417b-bb8f-88e9714f3cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.091880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.111940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-56e87d01-5360-417b-bb8f-88e9714f3cff None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:59.113280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-56e87d01-5360-417b-bb8f-88e9714f3cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:59.117209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 56/111] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:46:59.118695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.119238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.119838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.140710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:59.141977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:59.191209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 56/112] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:59.192638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.193001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.193351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.216672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:46:59.217949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:46:59.227262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 57/113] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:59.228389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.299295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 57/114] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:46:59.300309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.323510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.345644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56080ada-1ac5-44c8-b29c-9c674e34005b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.359182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.385179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.385994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.386281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.386557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.388103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:59.389291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:59.395471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 58/115] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:59.395471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.419997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50f538-8377-4444-b556-0b2276776a1e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.460803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.462244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.462563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.463548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.474568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:59.483439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:59.484433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:46:59.485367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:46:59.487321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 58/116] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:59.488265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.501388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82980921-dadc-48d2-996e-9262e02d0482 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.542114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.543089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.543436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.543786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.554657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:59.562803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:59.563856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:59.564816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 59/117] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:59.565542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.581357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.603787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad19b9e-ae0d-4c63-ba47-8a3f0bd196c8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.610722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.640813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.641629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.641902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.642176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.652935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:59.660491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:59.661299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 59/118] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:59.667790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.681121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226bcf8-ae3b-4a5b-93b7-c33aa4685d5f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.715312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.716148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.716436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.716713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.727705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:46:59.735377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:46:59.736366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:46:59.737275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 60/119] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:46:59.737980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.770587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4cca7-3dd6-42ea-8cee-3de3f83bc581 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.805155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.805823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.806261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.806665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.807742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:59.809029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:59.819077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.851900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 60/120] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:46:59 2022] POST /identity/v3/projects => generated 302 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:46:59.851900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.852181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8df17-dc67-412e-ac70-809c84a4d711 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:46:59.852810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:46:59.890481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:46:59.891169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:46:59.891451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:46:59.891886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:46:59.892800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:46:59.893113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:46:59.894597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:46:59.905194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 61/121] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:46:59.906288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:46:59.983633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179e0dd-63cc-4bfa-9dc9-0a520f1d3113 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.025744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59947e5a-0560-4a6b-b1df-0bd675f07e74 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.027032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.028216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.029589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.030091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.031056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:00.031585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.033326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.045255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 61/122] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.045255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.069948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.071012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.071491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.072109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.073069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:00.073608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.075217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.090360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 62/123] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:46:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.091271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.118827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.120195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.188556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cb798-7dca-4f7d-8389-b6b173d989ec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.216945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96500c8-3ab1-4831-a5d6-98a7d5e2a090 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.230574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.231265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.231546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.231820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.233028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:00.233342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.235081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.246415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 62/124] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.247538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.264646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.266199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.266712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.267178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.269480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.272611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.278098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 63/125] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.280053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.372906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fa39-a97b-46ee-9aab-707d90a43e75 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.388127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.409444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.410382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.410658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.410933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.412226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.412607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.413709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.415350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 63/126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.420904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.425551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f42e65-4fea-463d-9e40-8073e8427c5a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.470911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.472417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.473137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.473634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] PATH_INFO: `/v3/users/f55c7ea834cc41d8b2ddfaa12637822f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.495770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=f55c7ea834cc41d8b2ddfaa12637822f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.497855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.505263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 64/127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/users/f55c7ea834cc41d8b2ddfaa12637822f => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.505263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.548725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504f0fe-b975-48c5-bed4-0e83deacdacf None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.592897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.593874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.594221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.594572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.595832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.597466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.599244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 64/128] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.600133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.630280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b1af4-a8ba-44ec-91db-e0bd7665a1b3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.653728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.681652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.683507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.684012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.684459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.686573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.688575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.691346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 65/129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.692621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.708927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162110-f873-4853-b16e-a89fec5a4f9c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.714270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.744503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.745476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.745877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.746438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.747415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.748975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.751115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 65/130] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.756358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.803124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57daa8-71db-4e7c-bb40-26d2c8ce35e0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.841645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.842563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.843062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.843343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.859629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd219-a103-43a0-9174-aa6c6ebcd1de None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:00.876714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:00.894265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:00.895144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:00.895427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:00.895707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] PATH_INFO: `/v3/projects/4bb4a8a066894ff7974f0825bc04728d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:00.896920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] RBAC: Authorizing `identity:get_project(project_id=4bb4a8a066894ff7974f0825bc04728d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.898360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.900142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 66/131] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/projects/4bb4a8a066894ff7974f0825bc04728d => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:00.900883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:00.923441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:00.925024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:00.948380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:01.013602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527e9a-07f9-4a9e-af3d-8ebff7a46b86 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.032568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 66/132] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:01.033487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:01.059394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc477a63-91dd-4a85-afae-1375261a7f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:01.060339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc477a63-91dd-4a85-afae-1375261a7f8c None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.060623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc477a63-91dd-4a85-afae-1375261a7f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.060922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc477a63-91dd-4a85-afae-1375261a7f8c None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f/users/0e717e6887484c4c914aefb3e73d3210/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.063251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc477a63-91dd-4a85-afae-1375261a7f8c None admin] RBAC: Authorizing `identity:create_grant(project_id=f525943eff304b3786a90a844c39a37f, user_id=0e717e6887484c4c914aefb3e73d3210, role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:01.064757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc477a63-91dd-4a85-afae-1375261a7f8c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:01.092062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 67/133] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:00 2022] PUT /identity/v3/projects/f525943eff304b3786a90a844c39a37f/users/0e717e6887484c4c914aefb3e73d3210/roles/805a0b1c9f554aff8a16769124f939eb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:01.092062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fc2a2-62ed-4e02-8541-f1dcffdee66b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.092062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fc2a2-62ed-4e02-8541-f1dcffdee66b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.092062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fc2a2-62ed-4e02-8541-f1dcffdee66b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.092062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 68/134] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:01.092062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fc2a2-62ed-4e02-8541-f1dcffdee66b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:01.147117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6921023-42c1-4f41-a540-3f0f6a4e8b3e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.166859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:01.177506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:01.189177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:01.190149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.190432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.190710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f/users/7c92e7defa704085b3b2760be29aa346/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.193332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] RBAC: Authorizing `identity:create_grant(project_id=f525943eff304b3786a90a844c39a37f, user_id=7c92e7defa704085b3b2760be29aa346, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:01.194814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:01.215517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 67/135] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:01 2022] PUT /identity/v3/projects/f525943eff304b3786a90a844c39a37f/users/7c92e7defa704085b3b2760be29aa346/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:01.221091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:01.274397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fc2a2-62ed-4e02-8541-f1dcffdee66b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.324333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc593501-f1eb-41bc-af08-124c05e62f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:01.325383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc593501-f1eb-41bc-af08-124c05e62f37 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.325685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc593501-f1eb-41bc-af08-124c05e62f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.326118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc593501-f1eb-41bc-af08-124c05e62f37 None admin] PATH_INFO: `/v3/projects/4bb4a8a066894ff7974f0825bc04728d/users/0e717e6887484c4c914aefb3e73d3210/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.328349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc593501-f1eb-41bc-af08-124c05e62f37 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bb4a8a066894ff7974f0825bc04728d, user_id=0e717e6887484c4c914aefb3e73d3210, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:01.330027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc593501-f1eb-41bc-af08-124c05e62f37 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:01.350357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 69/136] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:01 2022] PUT /identity/v3/projects/4bb4a8a066894ff7974f0825bc04728d/users/0e717e6887484c4c914aefb3e73d3210/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:01.352022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.352383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.352803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.376399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:01.377850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:01.401489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c569e-b92c-408f-bac7-a9ee9a4ec600 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.457102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:01.458386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.458871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.459345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.460871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:01.462986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:01.465089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 68/137] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:01 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:01.471508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:01.472979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 70/138] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:01.511992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:01.532375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:01.572670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:01.594278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb197d89-659f-4b47-af34-efabc2df5770 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.645549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:01.648163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.649021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.654078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.655304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] RBAC: Adding query filter params (role.id=34639ea93eb34162ac3fe50f87d9e581, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=f55c7ea834cc41d8b2ddfaa12637822f) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:01.655834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:01.657731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:01.669611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 69/139] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:01 2022] GET /identity/v3/role_assignments?user.id=f55c7ea834cc41d8b2ddfaa12637822f&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=34639ea93eb34162ac3fe50f87d9e581 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:01.721893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf07d06-8cd3-45c9-bb7d-80f7f033aa64 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.779407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:01.780492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:01.780934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:01.781348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:01.801029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:01.809474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:01.810434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:01.811670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 71/140] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:01.816750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:01.962415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce27138-4a48-4a61-a1ca-7db3d047c8a9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:01.973031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:01.999637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:02.000704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.001221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.001700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.002651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:02.003159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:02.004802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:02.014784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 70/141] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:02.021099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:02.142897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7785-c8e3-4bbf-b60e-56d3524799cd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:02.144557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:02.183847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f452b-07ec-4a69-97c7-d15b0249767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:02.184814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f452b-07ec-4a69-97c7-d15b0249767b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.185098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f452b-07ec-4a69-97c7-d15b0249767b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.185370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f452b-07ec-4a69-97c7-d15b0249767b None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.186602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f452b-07ec-4a69-97c7-d15b0249767b None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:02.188111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f452b-07ec-4a69-97c7-d15b0249767b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:02.190067 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 72/142] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:02 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:02.195145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:02.213692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb8ee9-dac7-46ad-9969-af45d4df3417 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.214003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb8ee9-dac7-46ad-9969-af45d4df3417 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.214251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb8ee9-dac7-46ad-9969-af45d4df3417 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.215813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 73/143] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:02.226079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.226355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.226702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.252374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:02.253590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:02.395982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 74/144] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 30 09:47:02.411299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116e8eb-cf12-433a-ae53-8cbc50226275 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:02.450308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:02.451459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:02.452149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.452432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.452705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.510107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:02.556989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:02.558841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:02.702397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:02.715109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 71/145] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 09:47:02.720867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:02.725994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202827-36a5-482f-8fb7-9023f2cc69bd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:02.789458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:02.790270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.790601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.790873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.796586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:02.798061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:02.799710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 75/146] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:02.809190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:02.848347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:02.891896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58723438-f769-4844-945c-927c76a22fa8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:02.942611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:02.943928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:02.944445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:02.944918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:02.948717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:02.950203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:02.952004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 72/147] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:02 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:02.957592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:03.058216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25323efe-9fc6-499d-83e9-9b40e2ce1203 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:03.123396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1ffaa-3d02-4da7-be22-6967a7a645e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:03.124573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1ffaa-3d02-4da7-be22-6967a7a645e9 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:03.124573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1ffaa-3d02-4da7-be22-6967a7a645e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:03.124755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1ffaa-3d02-4da7-be22-6967a7a645e9 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:03.125412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1ffaa-3d02-4da7-be22-6967a7a645e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:03.126873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1ffaa-3d02-4da7-be22-6967a7a645e9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:03.167142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 76/148] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:02 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:03.176700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed33b-6a7b-411c-9d17-46ae11d26f50 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:03.231832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:03.233317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:03.233797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:03.234264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:03.235329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=0e717e6887484c4c914aefb3e73d3210) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:03.235923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:03.237596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b57c2b-e001-4f15-81f8-94a74a13ddbc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:03.244739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 73/149] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:02 2022] GET /identity/v3/role_assignments?user.id=0e717e6887484c4c914aefb3e73d3210&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:03.915308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517cf4-b2e6-44a3-ab5d-b0fce7a70eca None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:03.916393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517cf4-b2e6-44a3-ab5d-b0fce7a70eca None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:03.916877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517cf4-b2e6-44a3-ab5d-b0fce7a70eca None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:03.918479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 77/150] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:03.923997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bad86-907a-4347-b31d-3ee4d70bf378 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:03.924539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bad86-907a-4347-b31d-3ee4d70bf378 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:03.925023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bad86-907a-4347-b31d-3ee4d70bf378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:03.930839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:03.944385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-959bad86-907a-4347-b31d-3ee4d70bf378 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:03.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-959bad86-907a-4347-b31d-3ee4d70bf378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:04.026408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 74/151] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:04.056082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c517cf4-b2e6-44a3-ab5d-b0fce7a70eca None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:04.083763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:04.178727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c517cf4-b2e6-44a3-ab5d-b0fce7a70eca None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:04.217989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:04.218813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.219098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.219373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.230299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:04.234813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496d7dc-79b5-4b91-a81c-60bb68a0f125 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.235269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496d7dc-79b5-4b91-a81c-60bb68a0f125 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.235361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496d7dc-79b5-4b91-a81c-60bb68a0f125 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.237014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 75/152] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:04.238615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:04.239450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:04.240233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 78/153] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:04.245963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:04.246277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.246565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.246838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.283806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:04.286370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:04.389638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:04.401642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 76/154] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:04.427588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:04.435280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fd43a-0cce-439a-9f7f-e85c4d59c38c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:04.482771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:04.483456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.483734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.484031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.484807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:04.485108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:04.486627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e1774-ec4b-49a1-b79d-a8587538a67b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:04.498364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 79/155] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:04.499711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7610a-eb23-4e26-b007-daba8b37cb79 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.500054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7610a-eb23-4e26-b007-daba8b37cb79 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.500331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7610a-eb23-4e26-b007-daba8b37cb79 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.501691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 80/156] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:04.502771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8692b93-a08a-46c9-877a-29ad663afee3 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.502887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8692b93-a08a-46c9-877a-29ad663afee3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.503148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8692b93-a08a-46c9-877a-29ad663afee3 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.506797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 81/157] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:04.506797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8692b93-a08a-46c9-877a-29ad663afee3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:04.531256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:04.584460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc206a5-370f-4382-b177-222e837f54ff None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:04.630176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414c0d6-539b-4f1c-b908-1e254702c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:04.631828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414c0d6-539b-4f1c-b908-1e254702c9da None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.632597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414c0d6-539b-4f1c-b908-1e254702c9da None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.633062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414c0d6-539b-4f1c-b908-1e254702c9da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.635265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8692b93-a08a-46c9-877a-29ad663afee3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:04.646874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414c0d6-539b-4f1c-b908-1e254702c9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-f414c0d6-539b-4f1c-b908-1e254702c9da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:04.658088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:04.659243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:04.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:04.661357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 77/158] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:04.663960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.663960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.663960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.678763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9930c0-00eb-4a22-ac38-3bc7035f40c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:04.680036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9930c0-00eb-4a22-ac38-3bc7035f40c8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.680522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9930c0-00eb-4a22-ac38-3bc7035f40c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.681028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9930c0-00eb-4a22-ac38-3bc7035f40c8 None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.682444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9930c0-00eb-4a22-ac38-3bc7035f40c8 None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:04.684343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9930c0-00eb-4a22-ac38-3bc7035f40c8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:04.684520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:04.685771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:04.687040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 82/159] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:04 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:04.688603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.689099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.689611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.708272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:04.709452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:04.711608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:04.774423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 78/160] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:04.775925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:04.797641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 83/161] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:04.798508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:04.876432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:04.926847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2789f-b715-40ec-9f12-d549540fbefc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:04.964891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:04.965467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:04.965747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:04.966022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:04.984359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea89fe6-02f2-42eb-8702-63a48b41fc5a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:04.985642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:05.020712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.021812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.022354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.023083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.024155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:05.024665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.026507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.040023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 79/162] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:05.041679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.055077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.057187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.169178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491079aa-3d3e-4878-ae47-43038fdbafeb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.185787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:05.213277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.214091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.214370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.214634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.232371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 84/163] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:05.233158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.234104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:05.250853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:05.252229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 80/164] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:05.252995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.284562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:05.354698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4a0c7-7897-4eef-bed5-023fe2e78fa8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.402902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6c4cf-00b8-482b-b3d2-080cac90e639 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.410725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.411537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.411827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.412127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.433243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:05.437662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.438551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.438831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.439104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] PATH_INFO: `/v3/users/7c92e7defa704085b3b2760be29aa346` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.440319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] RBAC: Authorizing `identity:get_user(user_id=7c92e7defa704085b3b2760be29aa346)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.441779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.443712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 81/165] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/users/7c92e7defa704085b3b2760be29aa346 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:05.444506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:05.449089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:05.451106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 85/166] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:05.453045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.490860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:05.551359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701f24d-e533-4ae0-bebf-38cd59767174 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.587183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.588365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.588778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.589300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.590636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.592275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.594234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 82/167] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:05.595125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.643051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:05.651583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78f06-1fd8-40f7-aa5d-17ff5e72986c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.703095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b79985-e8db-4805-9717-40ea2a1b78e0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.714207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:05.714371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.715079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.715280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.715645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.716351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:05.716635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.718036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.735221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 86/168] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:05.736056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.752720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.753317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.753602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.753879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.859768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f004d3-1e45-4e22-b331-906fa678cf13 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:05.912256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.913736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.917909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:05.918588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:05.918863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:05.919134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:05.919899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:05.920208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:05.921579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:05.938441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 87/169] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:05.939116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:05.985125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:06.099396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 83/170] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:06.100271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.121532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:06.125466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1ff4c-f7d0-47d4-9f20-f5f5fa4cf847 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.189298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.189963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.190245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.190519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.191363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] RBAC: Adding query filter params (role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=0e717e6887484c4c914aefb3e73d3210) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:06.191673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.193152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.206478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 88/171] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:05 2022] GET /identity/v3/role_assignments?user.id=0e717e6887484c4c914aefb3e73d3210&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=805a0b1c9f554aff8a16769124f939eb => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:06.207310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.295484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a22938-eff1-43f1-a5a7-948785919581 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.370551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.371368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.371646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.371936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] PATH_INFO: `/v3/users/f55c7ea834cc41d8b2ddfaa12637822f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.373925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] RBAC: Authorizing `identity:get_user(user_id=f55c7ea834cc41d8b2ddfaa12637822f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.375402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.377090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 84/172] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:06 2022] GET /identity/v3/users/f55c7ea834cc41d8b2ddfaa12637822f => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:06.382521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.387914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:06.416820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e126b-9843-4d22-9767-333acc9e0545 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.501214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.502111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.502389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.502660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] PATH_INFO: `/v3/users/0e717e6887484c4c914aefb3e73d3210` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.508732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] RBAC: Authorizing `identity:get_user(user_id=0e717e6887484c4c914aefb3e73d3210)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.510363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.512332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 89/173] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:06 2022] GET /identity/v3/users/0e717e6887484c4c914aefb3e73d3210 => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:06.512953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.531959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:06.563586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba836fd6-3b56-44ce-baab-a0f9a7da9305 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.604436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.605733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.606009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.606289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.607582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.609560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.612297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 85/174] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:06 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:06.613275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.654205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33976d8-6718-45ad-a0eb-8cd3bd8b042e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.701523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac139c-1117-4009-8a53-231b78a41e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.702923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac139c-1117-4009-8a53-231b78a41e9f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.704270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac139c-1117-4009-8a53-231b78a41e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.704550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac139c-1117-4009-8a53-231b78a41e9f None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.706777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac139c-1117-4009-8a53-231b78a41e9f None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.708378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac139c-1117-4009-8a53-231b78a41e9f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.710115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 90/175] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:06 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:06.712601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60735e12-caa0-4f1b-abea-2fbc68de96bd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.712601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60735e12-caa0-4f1b-abea-2fbc68de96bd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.712905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60735e12-caa0-4f1b-abea-2fbc68de96bd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.714173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 91/176] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:06.715267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60735e12-caa0-4f1b-abea-2fbc68de96bd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.750483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645cc9-e35d-4ff2-821b-22ebc8941465 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.755072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:06.792541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d364054-211e-4fe9-82eb-d195ed7cb81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.793558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d364054-211e-4fe9-82eb-d195ed7cb81c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.794026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d364054-211e-4fe9-82eb-d195ed7cb81c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.794555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d364054-211e-4fe9-82eb-d195ed7cb81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.812582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:06.846672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60735e12-caa0-4f1b-abea-2fbc68de96bd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:06.871372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d364054-211e-4fe9-82eb-d195ed7cb81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.873224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d364054-211e-4fe9-82eb-d195ed7cb81c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.883997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:06.885479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.885965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.886481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.887514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=7c92e7defa704085b3b2760be29aa346) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:06.888064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:06.889969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:06.896541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 92/177] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:06 2022] GET /identity/v3/role_assignments?user.id=7c92e7defa704085b3b2760be29aa346&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:06.897554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:06.993136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 86/178] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:06.994649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8578-0755-42cb-ac48-9fe40f1b7777 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:06.994890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8578-0755-42cb-ac48-9fe40f1b7777 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:06.995168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8578-0755-42cb-ac48-9fe40f1b7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:06.998023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.015255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-dafd8578-0755-42cb-ac48-9fe40f1b7777 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:07.016133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c8b77-542c-4133-ab4b-ed29c633524b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.016494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dafd8578-0755-42cb-ac48-9fe40f1b7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:07.056107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b339f-40fd-4ef3-b165-17bd1e867b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.057578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b339f-40fd-4ef3-b165-17bd1e867b42 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.058057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b339f-40fd-4ef3-b165-17bd1e867b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.058534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b339f-40fd-4ef3-b165-17bd1e867b42 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f/users/f55c7ea834cc41d8b2ddfaa12637822f/roles/34639ea93eb34162ac3fe50f87d9e581` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.061370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b339f-40fd-4ef3-b165-17bd1e867b42 None admin] RBAC: Authorizing `identity:create_grant(project_id=f525943eff304b3786a90a844c39a37f, user_id=f55c7ea834cc41d8b2ddfaa12637822f, role_id=34639ea93eb34162ac3fe50f87d9e581)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.063179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b339f-40fd-4ef3-b165-17bd1e867b42 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.064571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.091358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 93/179] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:06 2022] PUT /identity/v3/projects/f525943eff304b3786a90a844c39a37f/users/f55c7ea834cc41d8b2ddfaa12637822f/roles/34639ea93eb34162ac3fe50f87d9e581 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:07.093122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62e81d-4299-4ab6-b39c-710231fa9abc None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.093423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62e81d-4299-4ab6-b39c-710231fa9abc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.093985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62e81d-4299-4ab6-b39c-710231fa9abc None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.095588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 94/180] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:07.100701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62e81d-4299-4ab6-b39c-710231fa9abc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.101268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 87/181] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:07.103306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdc37f-a8ab-44d4-8b80-98c96f0484d1 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.103692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdc37f-a8ab-44d4-8b80-98c96f0484d1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.104187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdc37f-a8ab-44d4-8b80-98c96f0484d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.122753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-edbdc37f-a8ab-44d4-8b80-98c96f0484d1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:07.124181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-edbdc37f-a8ab-44d4-8b80-98c96f0484d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:07.217034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 88/182] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:07.222890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd5dd-bf35-4617-a237-aa7072812512 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.223179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd5dd-bf35-4617-a237-aa7072812512 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.223461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd5dd-bf35-4617-a237-aa7072812512 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.224919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 89/183] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:07.229799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd5dd-bf35-4617-a237-aa7072812512 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.246729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62e81d-4299-4ab6-b39c-710231fa9abc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.293645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda386d-4689-4ce6-b5b4-1215b319cbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.294538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda386d-4689-4ce6-b5b4-1215b319cbde None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.294816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda386d-4689-4ce6-b5b4-1215b319cbde None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.295089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda386d-4689-4ce6-b5b4-1215b319cbde None admin] PATH_INFO: `/v3/projects/4bb4a8a066894ff7974f0825bc04728d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.296310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda386d-4689-4ce6-b5b4-1215b319cbde None admin] RBAC: Authorizing `identity:get_project(project_id=4bb4a8a066894ff7974f0825bc04728d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.297778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda386d-4689-4ce6-b5b4-1215b319cbde None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.299480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 95/184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/projects/4bb4a8a066894ff7974f0825bc04728d => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:07.305487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.305845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.306475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.323571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:07.324791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:07.334853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.350383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd5dd-bf35-4617-a237-aa7072812512 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.359129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.388587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.389383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.389665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.389936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.393156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 96/185] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:07.400132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.407464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:07.416631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:07.417528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:07.418300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 90/186] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:07.419317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.509140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6655f-3309-4fca-be2f-0118636cfa7d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.539824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8cf02-74c7-4321-84f3-5be5c8ce628f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.546332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.547159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.547450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.547729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.558812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:07.566956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:07.567830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:07.568745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 97/187] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:07.569375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.575446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.581387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.582063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.582345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.582620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.583467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=4bb4a8a066894ff7974f0825bc04728d, user.id=0e717e6887484c4c914aefb3e73d3210) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:07.583783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.585307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.599898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 91/188] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/role_assignments?user.id=0e717e6887484c4c914aefb3e73d3210&scope.project.id=4bb4a8a066894ff7974f0825bc04728d&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:07.599898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.642252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.705708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e703-ed03-4320-8f94-bfe5cd96ba16 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.743140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.743662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae2afd-3cd9-48df-a2d6-e66cc5f8e421 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.744011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.744292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.744564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.745334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:07.745640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.747085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.756611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 98/189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:07.757298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.788832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.789990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.790606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.790606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.792265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.793867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.795792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 92/190] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:07.796629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.847494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.886778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0cc81-7368-4867-92f3-b3b01dba0b96 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.910199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:07.914920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af6e9-28c3-47a3-816c-33630d75a2e0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:07.925808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.926492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.926768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.927037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.927789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:07.928120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.929510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.942283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 99/191] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:07.942283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:07.955693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:07.956678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:07.957021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:07.957358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] PATH_INFO: `/v3/users/f3d2545df6474980a8f697c6afa5f45c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:07.972517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d2545df6474980a8f697c6afa5f45c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:07.974371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:07.976297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 93/192] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/users/f3d2545df6474980a8f697c6afa5f45c => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:07.977027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.046082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898c9d3-cc51-442a-8736-dedb6f2f1807 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.075566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:08.082998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41a3ed-d142-4edc-a847-edd219bcd216 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.083955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.084636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.084918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.085187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.085909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.087824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.119901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 100/193] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:07 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:08.119901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.138601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.139645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.140023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.140357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] PATH_INFO: `/v3/users/f3d2545df6474980a8f697c6afa5f45c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.141845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d2545df6474980a8f697c6afa5f45c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.143502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.150812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 94/194] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:07 2022] GET /identity/v3/users/f3d2545df6474980a8f697c6afa5f45c => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:08.151725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.184766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:08.265613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2651f-56a5-4dd3-b163-6c9538f6558a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.291923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93c01-380e-45fb-956f-dc6ef685e090 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.316160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.317471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.317780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.318065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.333728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.334381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.334659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.334933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.391190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:08.413558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.417027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.426601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.428108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.466272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:08.548895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 101/195] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:08.554405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.555899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 95/196] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:08.561621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.646124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:08.664741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94ce60-629c-4a0a-b4a0-65af08b4ca51 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.699081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bc26a-3b6c-48a4-9e71-4fb09b0debba None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.707303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.708409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.709451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.709874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.712675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.714994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.718128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 102/197] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:08 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:08.726616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.729376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:08.749232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.750108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.750472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.750748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.752089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.753561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.755276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 96/198] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:08 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:08.761778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:08.906488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992940-04fd-40c8-8ae3-3d778043afc5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.945286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:08.946085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:08.946367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:08.946712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:08.947552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] RBAC: Adding query filter params (role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=f3d2545df6474980a8f697c6afa5f45c) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:08.947863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:08.949474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100eed6c-e4f7-4d59-9530-8780f50005c8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:08.949948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b477a8e-6b02-4a7d-b018-535762f71488 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:08.955751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 97/199] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:08 2022] GET /identity/v3/role_assignments?user.id=f3d2545df6474980a8f697c6afa5f45c&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=805a0b1c9f554aff8a16769124f939eb => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:09.006897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:09.007587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:09.007897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:09.008179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:09.009181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=f3d2545df6474980a8f697c6afa5f45c) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:09.009527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:09.011022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f808559-06b0-4323-9afe-c52cb0e606b9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:09.019122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 103/200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:08 2022] GET /identity/v3/role_assignments?user.id=f3d2545df6474980a8f697c6afa5f45c&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:09.249788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2070c7-9bf2-416f-924d-01e8f920bc97 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:09.249960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2070c7-9bf2-416f-924d-01e8f920bc97 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:09.250261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2070c7-9bf2-416f-924d-01e8f920bc97 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:09.252437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 98/201] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:09.257794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:09.258281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:09.258377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:09.281205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:09.282455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:09.295503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:09.366287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 104/202] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:09.413370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2070c7-9bf2-416f-924d-01e8f920bc97 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:09.490776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:09.546552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2070c7-9bf2-416f-924d-01e8f920bc97 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:09.594386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:09.595193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:09.595469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:09.595744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:09.612071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:09.619766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:09.620630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:09.621456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:09.624129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 99/203] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:09.626772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:09.747700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bc15a-8668-4e36-9e88-1c6b740e2485 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:09.788803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:09.789925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:09.790503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:09.790968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:09.791988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:09.792548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:09.794131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:09.808167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 105/204] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:09.813402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:09.967537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f21bd-e431-4977-90c2-121a2b7790a2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:09.989556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:10.013808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:10.014849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:10.015136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:10.015521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] PATH_INFO: `/v3/users/f55c7ea834cc41d8b2ddfaa12637822f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:10.016954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] RBAC: Authorizing `identity:get_user(user_id=f55c7ea834cc41d8b2ddfaa12637822f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:10.018417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:10.020332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 100/205] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:09 2022] GET /identity/v3/users/f55c7ea834cc41d8b2ddfaa12637822f => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:10.026398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:10.043454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:10.173980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cdada-02f6-478c-aee2-75fff3ca5aac None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:10.215628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:10.216613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:10.217070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:10.217516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:10.307510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:10.309704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:10.320235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:10.441375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 106/206] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:10.449723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:10.579954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d3c9a-8a46-4634-8883-971b5b9f01dc None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:10.629919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f82d9-8a15-4154-9833-6f46f430d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:10.630797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f82d9-8a15-4154-9833-6f46f430d004 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:10.631077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f82d9-8a15-4154-9833-6f46f430d004 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:10.631352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f82d9-8a15-4154-9833-6f46f430d004 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:10.632566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f82d9-8a15-4154-9833-6f46f430d004 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:10.634153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f82d9-8a15-4154-9833-6f46f430d004 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:10.635889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 101/207] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:10 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:10.641710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:10.754991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc76d9-9961-4fab-903e-acede4a5fe68 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:10.769997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:10.807432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:10.808133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:10.808417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:10.808691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:10.809550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] RBAC: Adding query filter params (role.id=34639ea93eb34162ac3fe50f87d9e581, scope.project.id=f525943eff304b3786a90a844c39a37f, user.id=f55c7ea834cc41d8b2ddfaa12637822f) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:10.809866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:10.811330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258c48-75ee-44bc-a20b-a90eb21eac5e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:10.820821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 107/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:10 2022] GET /identity/v3/role_assignments?user.id=f55c7ea834cc41d8b2ddfaa12637822f&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=34639ea93eb34162ac3fe50f87d9e581 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:10.823680 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379961d-7906-4816-8fd1-40806ff055c7 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:10.823994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379961d-7906-4816-8fd1-40806ff055c7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:10.824273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379961d-7906-4816-8fd1-40806ff055c7 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:10.825752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 102/209] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:10.831278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391c95b-e320-4b25-aaf7-47a2f87f2d63 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:10.831567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391c95b-e320-4b25-aaf7-47a2f87f2d63 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:10.831846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391c95b-e320-4b25-aaf7-47a2f87f2d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:10.854112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1391c95b-e320-4b25-aaf7-47a2f87f2d63 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:10.855323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1391c95b-e320-4b25-aaf7-47a2f87f2d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:10.951837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 108/210] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:10.977064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379961d-7906-4816-8fd1-40806ff055c7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.042194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:11.085091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc3de-40e1-466e-9c20-388c458b69ed None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.085477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc3de-40e1-466e-9c20-388c458b69ed None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.085641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc3de-40e1-466e-9c20-388c458b69ed None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.088007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 109/211] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:11.092745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.093026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.093300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.098702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379961d-7906-4816-8fd1-40806ff055c7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.114579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:11.115552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:11.137379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.138551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.139043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.140037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.141684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:11.143605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:11.150437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 103/212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:11.152321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.210319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 110/213] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:11.238143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.271296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:11.282143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef91dd-6ac6-4c0f-9366-3bb93152b442 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.305958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:11.320723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48395812-d941-4c7c-a1ae-d87fc7217f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.321577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48395812-d941-4c7c-a1ae-d87fc7217f96 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.321854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48395812-d941-4c7c-a1ae-d87fc7217f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.322128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48395812-d941-4c7c-a1ae-d87fc7217f96 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.322861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48395812-d941-4c7c-a1ae-d87fc7217f96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:11.325528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48395812-d941-4c7c-a1ae-d87fc7217f96 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:11.353765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b677b9a-bd0b-4ea8-b21e-91f94d04fb15 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.358580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 104/214] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:11 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:11.359997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05f8ee-857a-4c94-aca7-862edb95fc51 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.360350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05f8ee-857a-4c94-aca7-862edb95fc51 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.360675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05f8ee-857a-4c94-aca7-862edb95fc51 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.361994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 105/215] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:11.367804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.368167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.368490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.397504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:11.399215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:11.420573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.421571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.421973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.422259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.436705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:11.446605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:11.447797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:11.448946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 111/216] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:11.453894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.499917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 106/217] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:11.517660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.588338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:11.601496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2b584-efd6-4677-bf8e-0644ff78cdaf None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.643882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.644575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.644861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.645139 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.647585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:11.647585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:11.647585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:11.650050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4289-a730-417c-b1df-c5cc531e38b8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.657405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 112/218] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:11.663267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.663712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:11.700144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.701387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.701865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.702355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.715551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:11.728662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:11.729616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:11.730840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 107/219] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:11.736067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.793234 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50ac60-c0b5-4761-89f3-c581f80002b5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.831323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860d6b3-36a6-46d0-a348-3eb9f494b6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.832374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860d6b3-36a6-46d0-a348-3eb9f494b6ac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.832658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860d6b3-36a6-46d0-a348-3eb9f494b6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.832936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860d6b3-36a6-46d0-a348-3eb9f494b6ac None admin] PATH_INFO: `/v3/users/eecd9617ecf242caaa209efa790006dd` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.854009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860d6b3-36a6-46d0-a348-3eb9f494b6ac None admin] RBAC: Authorizing `identity:get_user(user_id=eecd9617ecf242caaa209efa790006dd)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:11.855300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860d6b3-36a6-46d0-a348-3eb9f494b6ac None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:11.857101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 113/220] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:11 2022] GET /identity/v3/users/eecd9617ecf242caaa209efa790006dd => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:11.862747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38861a1-5928-4325-a40f-8473890ec126 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.863042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38861a1-5928-4325-a40f-8473890ec126 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.863320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38861a1-5928-4325-a40f-8473890ec126 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.864772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 114/221] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:11.866230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38861a1-5928-4325-a40f-8473890ec126 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:11.884965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:11.897136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25d2f6-c895-4233-ae50-1817fe4dbc8e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:11.938878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:11.939893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.940189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.940465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.941499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:11.941808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:11.943669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891195-bea2-4dfd-a577-13662072ab62 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:11.955601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 108/222] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:11.957926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:11.958227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:11.958503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:11.979062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:11.980583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:11.994333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38861a1-5928-4325-a40f-8473890ec126 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.009887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.042542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.043821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.044333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.044980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.071895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 109/223] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:12.071895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.143445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.145509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.163778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.192582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d04864-6cee-4b3c-a0be-79c25b198309 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.232351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.233236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.233512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.233785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] PATH_INFO: `/v3/users/f3d2545df6474980a8f697c6afa5f45c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.235097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d2545df6474980a8f697c6afa5f45c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.236719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.242801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 110/224] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/users/f3d2545df6474980a8f697c6afa5f45c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:12.242801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.268057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 115/225] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 09:47:12.269246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.277095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.357912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dadef7-5788-4ce4-b2e9-8ff265016207 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.387161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977459c-c119-4bd5-8785-ab6dbd4f48da None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.394539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.395393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.395769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.396159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.403457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.409948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:12.418429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:12.419378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:12.420170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 111/226] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:12.421384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.429873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.431081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.431520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.431955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.433451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.435143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.442076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 116/227] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:12.443173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.513146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.530280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25024d6d-de11-4af0-8fb5-afe7c5c72c5b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.551210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b6cf-19f2-4e5d-af16-e84a515b30e5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.566059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.567162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.567540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.568035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.569326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.571105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.573266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 112/228] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:12.575251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.590394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.591374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.591957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.592412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.593704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:12.594187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.595887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.605897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 117/229] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:12.607026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.654047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.687324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2064948-0649-4d1a-bdad-c000e57b770c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.716584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066bf5f-9fce-4e4a-b87f-fd157893e42b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.723672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.725069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.725069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.725377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/f3d2545df6474980a8f697c6afa5f45c/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.727961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] RBAC: Authorizing `identity:create_grant(project_id=0760678cdad243f09545e4bf1c3266ba, user_id=f3d2545df6474980a8f697c6afa5f45c, role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.729681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.748146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 113/230] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:12 2022] PUT /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/f3d2545df6474980a8f697c6afa5f45c/roles/805a0b1c9f554aff8a16769124f939eb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:12.749855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:12.760258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.761409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.761887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.762351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.764029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=eecd9617ecf242caaa209efa790006dd) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:12.764538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.766480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.768818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:12.777214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 118/231] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/role_assignments?user.id=eecd9617ecf242caaa209efa790006dd&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:12.903407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2e29d-f5d9-4dfc-9d15-067088e96617 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:12.948853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:12.949689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:12.949972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:12.950315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] PATH_INFO: `/v3/users/f3d2545df6474980a8f697c6afa5f45c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:12.951594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] RBAC: Authorizing `identity:get_user(user_id=f3d2545df6474980a8f697c6afa5f45c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:12.953091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:12.955127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 114/232] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/users/f3d2545df6474980a8f697c6afa5f45c => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:12.960557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:13.082386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefbd5a-2b43-46f0-85c3-87a706e7d39d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:13.090182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:13.121677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6e89-4cdf-4db0-8dda-8b0bcfd5b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:13.122550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6e89-4cdf-4db0-8dda-8b0bcfd5b493 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:13.122832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6e89-4cdf-4db0-8dda-8b0bcfd5b493 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:13.123105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6e89-4cdf-4db0-8dda-8b0bcfd5b493 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:13.124321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc6e89-4cdf-4db0-8dda-8b0bcfd5b493 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:13.125784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc6e89-4cdf-4db0-8dda-8b0bcfd5b493 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:13.127781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 119/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:12 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:13.133555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:13.229087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:13.251139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b9c91-a03a-4b8e-8f9b-966e0b4d7ecb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:13.309760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e93b-766c-4749-9160-47345e98b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:13.310678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e93b-766c-4749-9160-47345e98b83b None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:13.310955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e93b-766c-4749-9160-47345e98b83b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:13.311224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e93b-766c-4749-9160-47345e98b83b None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/f3d2545df6474980a8f697c6afa5f45c/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:13.313204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889e93b-766c-4749-9160-47345e98b83b None admin] RBAC: Authorizing `identity:create_grant(project_id=0760678cdad243f09545e4bf1c3266ba, user_id=f3d2545df6474980a8f697c6afa5f45c, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:13.314668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889e93b-766c-4749-9160-47345e98b83b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:13.336279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 115/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:13 2022] PUT /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/f3d2545df6474980a8f697c6afa5f45c/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:13.832109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7226836-fd70-4454-bd29-2ec47d423ecd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:13.832662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7226836-fd70-4454-bd29-2ec47d423ecd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:13.832662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7226836-fd70-4454-bd29-2ec47d423ecd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:13.834274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 120/235] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:13.841553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cafa5-6386-43cc-a4ed-6db7802f7c86 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:13.841966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cafa5-6386-43cc-a4ed-6db7802f7c86 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:13.842138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cafa5-6386-43cc-a4ed-6db7802f7c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:13.864368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-be5cafa5-6386-43cc-a4ed-6db7802f7c86 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:13.865547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-be5cafa5-6386-43cc-a4ed-6db7802f7c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:13.953280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 116/236] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:13.970644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7075e-29f9-4806-9ca6-120b726a4b70 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:13.970922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7075e-29f9-4806-9ca6-120b726a4b70 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:13.971198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7075e-29f9-4806-9ca6-120b726a4b70 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:13.972604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 121/237] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:13.977607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:13.977908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:13.978180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:13.981124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7075e-29f9-4806-9ca6-120b726a4b70 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:14.004377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:14.005568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:14.130408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 117/238] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:14.137596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7075e-29f9-4806-9ca6-120b726a4b70 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:14.184004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:14.188209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:14.198797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:14.199627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:14.199924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:14.200202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:14.222038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:14.238289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:14.239239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:14.240506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 122/239] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:14.254202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:14.312911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15968fa-72d7-4a27-a35c-6151e2dceffe None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:14.328216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:14.376275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:14.377090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:14.377371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:14.377642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:14.379000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:14.382982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:14.385019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 118/240] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:14.392145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:14.451065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461c874-ed02-4380-8749-02d17d76d3f2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:14.516695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:14.518625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:14.518914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:14.519191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:14.521052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:14.521366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:14.525741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:14.526185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:14.542374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 123/241] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:14.550564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:14.583420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f7ea-f87a-4c56-b96e-6da7ad8a7056 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:14.635426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:14.636081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:14.636367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:14.636642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:14.637385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:14.638842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:14.679158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 119/242] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:14 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:14.681509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:14.696457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4954322-e4fc-4d4a-a277-50e7a406e7d3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:14.757795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:14.758719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:14.759081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:14.759394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] PATH_INFO: `/v3/users/c9078c7fe26f46eaab771a289a4e95d9` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:14.784843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] RBAC: Authorizing `identity:get_user(user_id=c9078c7fe26f46eaab771a289a4e95d9)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:14.786422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:14.788376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 124/243] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:14 2022] GET /identity/v3/users/c9078c7fe26f46eaab771a289a4e95d9 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:14.794419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:14.908470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838836f3-7c83-426d-a9b3-3c3c192e066c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:14.937147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:14.945823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095bfc9-1fba-4bb4-9e8a-828233ff17fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:14.946569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095bfc9-1fba-4bb4-9e8a-828233ff17fe None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:14.946932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095bfc9-1fba-4bb4-9e8a-828233ff17fe None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:14.947124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095bfc9-1fba-4bb4-9e8a-828233ff17fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.019785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095bfc9-1fba-4bb4-9e8a-828233ff17fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:15.021342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095bfc9-1fba-4bb4-9e8a-828233ff17fe None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:15.124232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 120/244] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:15.129937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:15.234231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5966e5-367b-472e-a11c-4a0ca3966d07 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:15.255149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1322e3-197d-418e-b6c2-6f55cd8586a3 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.255466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1322e3-197d-418e-b6c2-6f55cd8586a3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.255701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1322e3-197d-418e-b6c2-6f55cd8586a3 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.257190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 121/245] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:15.263640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.263995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.264225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.284738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:15.285942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:15.293610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:15.294866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.295353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.295813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.297333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:15.299006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:15.300958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 125/246] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:15 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:15.308769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:15.322568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:15.351576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:15.387542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 122/247] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:15.411402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:15.451450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d13e8-a3af-4e98-924b-cb346c870bbe None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:15.497422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:15.498413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.498886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.499443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.500537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] RBAC: Adding query filter params (role.id=34639ea93eb34162ac3fe50f87d9e581, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=c9078c7fe26f46eaab771a289a4e95d9) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:15.501490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:15.503230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9f8c6-ec80-4672-9807-bb2d846441fb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:15.511588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 126/248] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:15 2022] GET /identity/v3/role_assignments?user.id=c9078c7fe26f46eaab771a289a4e95d9&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=34639ea93eb34162ac3fe50f87d9e581 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:15.513803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede9c2d-6303-44ed-9064-f9b00d740341 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.514323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede9c2d-6303-44ed-9064-f9b00d740341 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.514788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede9c2d-6303-44ed-9064-f9b00d740341 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.516982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 127/249] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:15.523483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.524186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.526571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.537515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6aee5-2c03-48c5-9026-e829c5b56e18 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:15.548277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:15.549833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:15.632343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:15.633590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.634344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.635899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.653677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:15.662219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 128/250] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:15.662869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:15.663709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:15.666519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:15.669138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 123/251] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:15.674765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:15.702336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:15.723675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:15.735898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:15.855372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6422145-4256-4fa2-8fe1-f6a8f37ff017 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:15.890075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a1bbd-4224-43a6-976b-2b96cdec9fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:15.891252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a1bbd-4224-43a6-976b-2b96cdec9fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.891789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a1bbd-4224-43a6-976b-2b96cdec9fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.892272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a1bbd-4224-43a6-976b-2b96cdec9fe1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.897999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2bee3-1d65-418f-852b-af52d2a4bdb2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:15.903440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a1bbd-4224-43a6-976b-2b96cdec9fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-661a1bbd-4224-43a6-976b-2b96cdec9fe1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:15.913164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:15.914509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:15.915464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:15.918338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 124/252] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:15.919833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b65d-f534-4197-a269-ad2ecf30abd2 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:15.920334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b65d-f534-4197-a269-ad2ecf30abd2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:15.920812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b65d-f534-4197-a269-ad2ecf30abd2 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:15.922544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 125/253] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:15.924298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00b65d-f534-4197-a269-ad2ecf30abd2 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.008992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.009506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.009789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.010067 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.010840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:16.011169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.011815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.012569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f40e39-1827-4865-aa58-62a772985627 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.034285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 129/254] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:16.035526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.035898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.036189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.037560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00b65d-f534-4197-a269-ad2ecf30abd2 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.075451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.076693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.077166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.077621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.078559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:16.079145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.080773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.084895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:16.086095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:16.091758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 126/255] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:16.093854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.205384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb8553-beb9-48d5-a664-bbd62817e2f0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.226752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 130/256] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:16.227666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.244360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.245571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.246112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.246616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] PATH_INFO: `/v3/users/f3d2545df6474980a8f697c6afa5f45c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.248094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d2545df6474980a8f697c6afa5f45c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.249738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.249856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.252502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 127/257] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/users/f3d2545df6474980a8f697c6afa5f45c => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:16.261259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.283261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.373761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b387e-541a-4a06-a1d0-10f925a112f1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.416227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.417123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.417399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.417671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] PATH_INFO: `/v3/users/eecd9617ecf242caaa209efa790006dd` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.418874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] RBAC: Authorizing `identity:get_user(user_id=eecd9617ecf242caaa209efa790006dd)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.420373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.422300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 131/258] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/users/eecd9617ecf242caaa209efa790006dd => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:16.423082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.489867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9ac3-2c11-445d-98c0-9d650f8ff713 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.502279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.527348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1d791-5dee-49ba-bdef-ecea09a897f6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.561640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.562581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.562908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.563181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.569481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.570150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.570439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.570715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.573867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:16.581741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:16.582630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 132/259] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:16.584951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.691999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b20de-53ab-4d34-adf9-891649ad2465 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.698396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.700170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.721333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.722255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.725885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.725885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.725885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.725885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.726304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.728393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 133/260] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:16.729158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.733296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.841230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13d7d-41da-4f77-94cc-e3a3f5975e2a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:16.875988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:16.876668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:16.876946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:16.877217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:16.877968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:16.878270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:16.879659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:16.888461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 134/261] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:16.889245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.950643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 128/262] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:16.951499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:16.956311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:16.996066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baef48-7136-43f5-a97b-bda95feb73d6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:17.032613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:17.038849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:17.039131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:17.039407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/eecd9617ecf242caaa209efa790006dd/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:17.041535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] RBAC: Authorizing `identity:create_grant(project_id=0760678cdad243f09545e4bf1c3266ba, user_id=eecd9617ecf242caaa209efa790006dd, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:17.042999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:17.066052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 135/263] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:16 2022] PUT /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/eecd9617ecf242caaa209efa790006dd/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:17.067380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:17.180654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f6737-223e-4b9a-a26d-0f778516527c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:17.200542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:17.268042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:17.268915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:17.269193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:17.269465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] PATH_INFO: `/v3/users/f3d2545df6474980a8f697c6afa5f45c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:17.275108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d2545df6474980a8f697c6afa5f45c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:17.276643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:17.282415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 129/264] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:16 2022] GET /identity/v3/users/f3d2545df6474980a8f697c6afa5f45c => generated 283 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:17.284312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:17.310146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943f8b-5a87-4976-ad85-926374da5989 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:17.387856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:17.410436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:17.411499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:17.411895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:17.412271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:17.413024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e296a4-ff6b-4695-b726-a4f239db4287 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:17.413672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:17.415415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:17.417496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 136/265] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:17 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:17.423359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:17.454334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:17.455227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:17.455659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:17.456091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:17.511467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:17.543953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:17.546255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:17.547656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf15f77-d934-4ad5-8168-890f69070b29 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:17.608995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:17.609853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:17.610232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:17.610603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:17.611637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] RBAC: Adding query filter params (role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=f3d2545df6474980a8f697c6afa5f45c) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:17.612071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:17.613745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:17.628535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 137/266] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:17 2022] GET /identity/v3/role_assignments?user.id=f3d2545df6474980a8f697c6afa5f45c&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=805a0b1c9f554aff8a16769124f939eb => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:17.661076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 130/267] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:17.667132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:17.769184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:17.820820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0514db-06dc-45bd-b47c-e93819ad63a8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:17.863060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2d912-a714-4a26-b88a-fc671a509781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:17.863968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2d912-a714-4a26-b88a-fc671a509781 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:17.864256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2d912-a714-4a26-b88a-fc671a509781 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:17.864528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2d912-a714-4a26-b88a-fc671a509781 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:17.865727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2d912-a714-4a26-b88a-fc671a509781 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:17.867188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2d912-a714-4a26-b88a-fc671a509781 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:17.869326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 138/268] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:17 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:17.875653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:17.967914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:17.997753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87fa5d-32f5-4102-809f-1bd0daf17dd0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.034666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.035297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.035575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.035844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.036713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=f3d2545df6474980a8f697c6afa5f45c) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:18.037021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:18.038476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031bef-9c6f-49f0-93ee-5bdf731fed71 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:18.045255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 131/269] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:17 2022] GET /identity/v3/role_assignments?user.id=f3d2545df6474980a8f697c6afa5f45c&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:18.052531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfffc8-0237-41d0-affd-a647b4b3d3a5 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.052819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfffc8-0237-41d0-affd-a647b4b3d3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.053096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfffc8-0237-41d0-affd-a647b4b3d3a5 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.054932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 139/270] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:18.060817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.061132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.061376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.081876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:18.083083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:18.148729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23111360-7558-42d4-b4af-9541cee3a45e None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.149060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23111360-7558-42d4-b4af-9541cee3a45e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.149279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23111360-7558-42d4-b4af-9541cee3a45e None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.150821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 140/271] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:18.156337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227e66c-cc45-4cf8-a328-4c885faf981d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.156618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227e66c-cc45-4cf8-a328-4c885faf981d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.156892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227e66c-cc45-4cf8-a328-4c885faf981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.187327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1227e66c-cc45-4cf8-a328-4c885faf981d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:18.192200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1227e66c-cc45-4cf8-a328-4c885faf981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:18.198255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 132/272] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:18.237074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.303203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 141/273] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:18.305065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febab18-2ddc-4b04-b6d2-2cfe72a05b28 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.305558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febab18-2ddc-4b04-b6d2-2cfe72a05b28 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.306055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febab18-2ddc-4b04-b6d2-2cfe72a05b28 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.307999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 142/274] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:18.313689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.314267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.314267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.336444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:18.341768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:18.343079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:18.378653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65435d89-cef8-4521-bf34-12a41267e1f5 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.408247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:18.422325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.423403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.423862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.424165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.432406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 143/275] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:18.433313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.436415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:18.447442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:18.448382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:18.449164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 133/276] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:18.455700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.550413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c756ac-6535-44a9-8bc8-24d352957d46 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.566078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7acdd6-d3cc-42ba-b1db-55b527a893aa None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.584242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.585041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.585350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.585578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.596020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:18.597108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:18.604894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:18.605804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:18.606597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 144/277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:18.607256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.613612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2d595-9744-4922-b475-891013755bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.614300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2d595-9744-4922-b475-891013755bac None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.614586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2d595-9744-4922-b475-891013755bac None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.614861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2d595-9744-4922-b475-891013755bac None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.615632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2d595-9744-4922-b475-891013755bac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:18.615978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2d595-9744-4922-b475-891013755bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:18.617810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2d595-9744-4922-b475-891013755bac None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:18.628426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 134/278] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:18.629165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2d595-9744-4922-b475-891013755bac None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.658999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:18.716513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768e862-f92f-4c95-b196-1515a0fb9d61 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.736156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2d595-9744-4922-b475-891013755bac None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.750303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.751113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.751397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.751674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.752975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:18.754496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:18.759920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 145/279] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:18.759920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.780619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.781743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.782027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.782454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.783205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:18.783521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:18.784945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:18.793706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 135/280] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:18.794505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.852330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:18.869959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a32513-1c91-459b-b892-a687a6107ab2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.886286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:18.910544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.911456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63690db9-71e6-4660-bf71-e531a0e65b97 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:18.911971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.912423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.912892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] PATH_INFO: `/v3/users/c9078c7fe26f46eaab771a289a4e95d9` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.914268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] RBAC: Authorizing `identity:get_user(user_id=c9078c7fe26f46eaab771a289a4e95d9)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:18.916108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:18.923028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 146/281] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/users/c9078c7fe26f46eaab771a289a4e95d9 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:18.924132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:18.957622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:18.958311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:18.958600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:18.959148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:18.960321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:18.962615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:18.999903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 136/282] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:18 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:18.999903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:19.100776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b56a-da2f-4262-b59b-c5412a182a0b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:19.175927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:19.176130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:19.176346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:19.176615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] PATH_INFO: `/v3/users/8030ea88088047e1b71ca27f8b47fe35` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:19.193342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] RBAC: Authorizing `identity:get_user(user_id=8030ea88088047e1b71ca27f8b47fe35)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:19.194915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:19.197793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 147/283] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/users/8030ea88088047e1b71ca27f8b47fe35 => generated 293 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:19.203745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:19.207425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b46d9-c004-400d-943e-2ffb8409f682 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:19.218452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:19.270566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:19.271833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:19.272653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:19.273118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:19.277252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:19.278820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:19.281599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 137/284] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:18 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:19.286107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:19.286289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:19.386943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100bb2e-867b-4a28-b5ab-81bcd614b10f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:19.431415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:19.432536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:19.433144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:19.433627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:19.434649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, user.id=8030ea88088047e1b71ca27f8b47fe35, scope.system=all) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:19.435146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:19.435762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb653311-b1e0-4183-ae19-60a8c370c5b6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:19.437265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01261d-5731-4ed2-9884-e8f5d5b20cb4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:19.443275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 148/285] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 09:47:19 2022] GET /identity/v3/role_assignments?user.id=8030ea88088047e1b71ca27f8b47fe35&scope.system=all&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:19.498336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2c5d8-63b6-4e38-9906-31226052e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:19.499656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2c5d8-63b6-4e38-9906-31226052e0cd None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:19.500184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2c5d8-63b6-4e38-9906-31226052e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:19.500648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2c5d8-63b6-4e38-9906-31226052e0cd None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/c9078c7fe26f46eaab771a289a4e95d9/roles/34639ea93eb34162ac3fe50f87d9e581` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:19.503028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2c5d8-63b6-4e38-9906-31226052e0cd None admin] RBAC: Authorizing `identity:create_grant(project_id=0760678cdad243f09545e4bf1c3266ba, user_id=c9078c7fe26f46eaab771a289a4e95d9, role_id=34639ea93eb34162ac3fe50f87d9e581)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:19.504878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2c5d8-63b6-4e38-9906-31226052e0cd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:19.524410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 138/286] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:19 2022] PUT /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba/users/c9078c7fe26f46eaab771a289a4e95d9/roles/34639ea93eb34162ac3fe50f87d9e581 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 09:47:19.728918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a09fd-d76e-47a0-9e23-284c03a55204 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:19.729571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a09fd-d76e-47a0-9e23-284c03a55204 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:19.730064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a09fd-d76e-47a0-9e23-284c03a55204 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:19.731630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 149/287] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:19.737210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:19.737497 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:19.737772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:19.757269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:19.758478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:19.789683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:19.847172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 139/288] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:19.882390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a09fd-d76e-47a0-9e23-284c03a55204 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:20.016219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a09fd-d76e-47a0-9e23-284c03a55204 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:20.018576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:20.061525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:20.062333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:20.062614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:20.062891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:20.074934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:20.082495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:20.083360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:20.084189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 150/289] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:20.091794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:20.235750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18893cda-07c8-4877-9579-b8de1402fa2d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:20.271545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:20.272174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:20.272459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:20.272734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:20.273498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:20.273804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:20.275915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:20.285078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 140/290] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:20.291148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:20.406375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731e279-6d50-4878-a59a-30f471665541 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:20.448155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:20.449052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:20.449335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:20.449611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] PATH_INFO: `/v3/users/eecd9617ecf242caaa209efa790006dd` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:20.451316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] RBAC: Authorizing `identity:get_user(user_id=eecd9617ecf242caaa209efa790006dd)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:20.453111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:20.455146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 151/291] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:20 2022] GET /identity/v3/users/eecd9617ecf242caaa209efa790006dd => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:20.460478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:20.510624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:20.599467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e3301-15cf-4a7d-a695-2f5b19d325dd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:20.646316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:20.646976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:20.647264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:20.647538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:20.748317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:20.749848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:20.794495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:20.889146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 141/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 09:47:20.895015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:20.948238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:21.060513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6debe-8e3f-41f2-ad31-520a7228b2ef None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:21.105093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861747ae-01aa-43c7-b756-591ecc0e2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:21.106340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861747ae-01aa-43c7-b756-591ecc0e2e01 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.106811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861747ae-01aa-43c7-b756-591ecc0e2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.107272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861747ae-01aa-43c7-b756-591ecc0e2e01 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.108695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861747ae-01aa-43c7-b756-591ecc0e2e01 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:21.110390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861747ae-01aa-43c7-b756-591ecc0e2e01 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:21.112318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 152/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:20 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:21.118665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:21.256278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa5ed-7cfa-4a90-933f-a4d9a137d8be None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:21.303904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:21.304755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.305036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.305309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.306198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=eecd9617ecf242caaa209efa790006dd) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:21.306512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:21.308034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115aa09-966d-4391-a71d-3cd9b7590164 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:21.316117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:21.320013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 142/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:21 2022] GET /identity/v3/role_assignments?user.id=eecd9617ecf242caaa209efa790006dd&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:21.697307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985068c4-d732-4563-b086-85793f20a230 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.698343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985068c4-d732-4563-b086-85793f20a230 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.698628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985068c4-d732-4563-b086-85793f20a230 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.700411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 153/295] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:21.705612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaca57-cb2a-4a2c-950e-6f91b482cd36 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.705899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaca57-cb2a-4a2c-950e-6f91b482cd36 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.706176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaca57-cb2a-4a2c-950e-6f91b482cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.728373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9aaaca57-cb2a-4a2c-950e-6f91b482cd36 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:21.729564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9aaaca57-cb2a-4a2c-950e-6f91b482cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:21.812558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 143/296] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:21.839526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985068c4-d732-4563-b086-85793f20a230 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:21.852222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013a357-f7ed-4b2b-ad98-56f35afbfa21 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.852615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013a357-f7ed-4b2b-ad98-56f35afbfa21 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.852842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013a357-f7ed-4b2b-ad98-56f35afbfa21 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.854511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 144/297] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:21.860452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd279a-8594-4e17-83a9-76cd2b7ac8a3 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.860736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd279a-8594-4e17-83a9-76cd2b7ac8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.861008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd279a-8594-4e17-83a9-76cd2b7ac8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.881722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bbbd279a-8594-4e17-83a9-76cd2b7ac8a3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:21.883552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bbbd279a-8594-4e17-83a9-76cd2b7ac8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:21.956180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:21.960395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985068c4-d732-4563-b086-85793f20a230 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:21.962864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 145/298] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:21.965227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba9de8-14e0-4bf9-82da-a48040ea6472 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.965511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba9de8-14e0-4bf9-82da-a48040ea6472 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.965785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba9de8-14e0-4bf9-82da-a48040ea6472 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.967198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 146/299] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:21.974924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:21.974924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:21.974924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:21.993261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:21.994438 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:22.002068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.002871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.003150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.003422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.015216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:22.022751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:22.023699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 154/300] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:22.029985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.054592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.075332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 147/301] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:22.076120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.141489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d60bad-6854-4a77-9c86-51bcc8e71b6d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.176593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.177389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.177668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.177936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.179092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.180634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.186677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 155/302] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:22.186677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.186867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e7438-ae08-4577-abc7-66eff151ade0 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.216803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.220052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.220738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.221017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.221289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.222122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:22.222432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.223826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.233482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 148/303] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:22.234205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.281994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.294356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73cdb4-aa96-4f4d-9aaf-3026d477c853 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.326694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.327490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.327773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.328079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.334390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14219cf-c482-4e16-9c54-cc6f00dc0db3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.338399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:22.345981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:22.346902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:22.347682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 156/304] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:22.348446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.374061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cc36-fafc-4c04-aff6-5cd43750de01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.374738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cc36-fafc-4c04-aff6-5cd43750de01 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.374991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cc36-fafc-4c04-aff6-5cd43750de01 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.375267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cc36-fafc-4c04-aff6-5cd43750de01 None admin] PATH_INFO: `/v3/groups` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.376019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cc36-fafc-4c04-aff6-5cd43750de01 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.377466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cc36-fafc-4c04-aff6-5cd43750de01 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.391966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 149/305] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Jun 30 09:47:22 2022] POST /identity/v3/groups => generated 229 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:22.393229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f976d2-a29b-4e38-ae24-9fb28b20d8d7 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.393514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f976d2-a29b-4e38-ae24-9fb28b20d8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.393785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f976d2-a29b-4e38-ae24-9fb28b20d8d7 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.395069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 150/306] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:22.395774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f976d2-a29b-4e38-ae24-9fb28b20d8d7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.457576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.471477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f3766-cbfc-4295-951d-72225bbe692c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.508228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a0727-f850-4c00-b8af-d18c2432e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.509779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a0727-f850-4c00-b8af-d18c2432e801 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.510673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a0727-f850-4c00-b8af-d18c2432e801 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.511300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a0727-f850-4c00-b8af-d18c2432e801 None admin] PATH_INFO: `/v3/users/629c17862f6741ff84f18939fc7c2fc2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.520346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f976d2-a29b-4e38-ae24-9fb28b20d8d7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.523416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.530015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a0727-f850-4c00-b8af-d18c2432e801 None admin] RBAC: Authorizing `identity:get_user(user_id=629c17862f6741ff84f18939fc7c2fc2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.534386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a0727-f850-4c00-b8af-d18c2432e801 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.541142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 157/307] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/users/629c17862f6741ff84f18939fc7c2fc2 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:22.541142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29506507-18da-4883-9e40-2034a14c6453 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.541142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29506507-18da-4883-9e40-2034a14c6453 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.541142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29506507-18da-4883-9e40-2034a14c6453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.555794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-29506507-18da-4883-9e40-2034a14c6453 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:22.557368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-29506507-18da-4883-9e40-2034a14c6453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:22.571499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.572352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.572635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.572905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.573663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:22.573971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.575377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.588029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 151/308] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:22.589057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.628980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 158/309] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:22.630491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29506507-18da-4883-9e40-2034a14c6453 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.697911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34006-98ac-4442-8693-47f41ac56d35 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.725405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.735945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.736901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.737164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.737437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.738271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] RBAC: Adding query filter params (role.id=34639ea93eb34162ac3fe50f87d9e581, user.id=629c17862f6741ff84f18939fc7c2fc2, scope.system=all) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:22.738584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.740516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.752196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 152/310] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/role_assignments?user.id=629c17862f6741ff84f18939fc7c2fc2&scope.system=all&role.id=34639ea93eb34162ac3fe50f87d9e581 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:22.752196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.763328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29506507-18da-4883-9e40-2034a14c6453 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.789209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:22.805294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.806296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.806573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.806848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] PATH_INFO: `/v3/users/8030ea88088047e1b71ca27f8b47fe35` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.808196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] RBAC: Authorizing `identity:get_user(user_id=8030ea88088047e1b71ca27f8b47fe35)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.809677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.811786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 159/311] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/users/8030ea88088047e1b71ca27f8b47fe35 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:22.817552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.887066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe3ec0-741e-479a-8f90-24e787a81a62 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.927717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.928581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.928862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.929138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.933600 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe3dfa-4e18-4dab-b075-6c0eebffbfd9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:22.943799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:22.952614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:22.953510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:22.954310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 153/312] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:22.959748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:22.979112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:22.979980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:22.980228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:22.980502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] PATH_INFO: `/v3/system/users/8030ea88088047e1b71ca27f8b47fe35/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:22.982287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8030ea88088047e1b71ca27f8b47fe35, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:22.983620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:22.997833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 160/313] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Jun 30 09:47:22 2022] PUT /identity/v3/system/users/8030ea88088047e1b71ca27f8b47fe35/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 09:47:23.077235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:23.089774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c96d26-9574-4865-976a-e44cbed7b9b3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:23.132637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:23.133332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:23.133677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:23.133978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:23.134786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:23.135222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:23.136675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:23.146793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 154/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:23.153644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:23.166792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:23.261823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05282a-7e36-47dd-99cf-594d06b49e54 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:23.302584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:23.303655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:23.304101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:23.304530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] PATH_INFO: `/v3/users/c9078c7fe26f46eaab771a289a4e95d9` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:23.306138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] RBAC: Authorizing `identity:get_user(user_id=c9078c7fe26f46eaab771a289a4e95d9)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:23.307924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:23.310007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 161/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:23 2022] GET /identity/v3/users/c9078c7fe26f46eaab771a289a4e95d9 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:23.315358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:23.426343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d770-478c-455f-b387-f08e1ff0e501 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:23.466080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:23.466744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:23.467025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:23.467300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:23.489872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:23.537118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:23.538528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:23.646665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 155/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 09:47:23.652453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:23.740120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:23.768428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23359189-bd37-480e-897c-548865c4ea4d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:23.815946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb947b-21ae-4208-848d-25fc8450a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:23.816751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb947b-21ae-4208-848d-25fc8450a154 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:23.817034 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb947b-21ae-4208-848d-25fc8450a154 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:23.817324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb947b-21ae-4208-848d-25fc8450a154 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:23.818565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb947b-21ae-4208-848d-25fc8450a154 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:23.820097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb947b-21ae-4208-848d-25fc8450a154 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:23.822269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 162/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:23 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:23.827863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:23.890771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:23.949707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b6731-3aa7-4842-bed8-3dd86b2f997d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:23.994288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:23.995024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:23.995304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:23.995624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:23.996555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] RBAC: Adding query filter params (role.id=34639ea93eb34162ac3fe50f87d9e581, scope.project.id=0760678cdad243f09545e4bf1c3266ba, user.id=c9078c7fe26f46eaab771a289a4e95d9) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:23.996983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:23.998500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb51d77-1c80-4e7c-8d59-19afe771a7f6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:24.007673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 156/318] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:23 2022] GET /identity/v3/role_assignments?user.id=c9078c7fe26f46eaab771a289a4e95d9&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=34639ea93eb34162ac3fe50f87d9e581 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:24.488435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861d7e5-9a2e-457d-b997-031180efb0c9 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:24.488834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861d7e5-9a2e-457d-b997-031180efb0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:24.489001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861d7e5-9a2e-457d-b997-031180efb0c9 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:24.490540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 163/319] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:24.496083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:24.496601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:24.496601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:24.514874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:24.516169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:24.589079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 157/320] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:24.611610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861d7e5-9a2e-457d-b997-031180efb0c9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:24.725811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861d7e5-9a2e-457d-b997-031180efb0c9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:24.760350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcdf07-dd0d-4d63-869a-57fc78faa1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:24.761136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcdf07-dd0d-4d63-869a-57fc78faa1cf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:24.761434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcdf07-dd0d-4d63-869a-57fc78faa1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:24.761709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcdf07-dd0d-4d63-869a-57fc78faa1cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:24.762891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcdf07-dd0d-4d63-869a-57fc78faa1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:24.764586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcdf07-dd0d-4d63-869a-57fc78faa1cf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:24.766390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 164/321] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:24.772037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:24.822534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:24.880746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c15ac-4354-4c26-85e7-9f6c51a35564 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:24.916223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252b96-1fa8-4595-af0b-14b7f2d1c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:24.916905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252b96-1fa8-4595-af0b-14b7f2d1c0ac None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:24.917184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252b96-1fa8-4595-af0b-14b7f2d1c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:24.917453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252b96-1fa8-4595-af0b-14b7f2d1c0ac None admin] PATH_INFO: `/v3/groups` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:24.918168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252b96-1fa8-4595-af0b-14b7f2d1c0ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:24.919616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252b96-1fa8-4595-af0b-14b7f2d1c0ac None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:24.935063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 158/322] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Jun 30 09:47:24 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:24.955005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1442-ee8a-4d59-ba0a-d4c49c6fce62 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:24.955351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1442-ee8a-4d59-ba0a-d4c49c6fce62 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:24.955554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1442-ee8a-4d59-ba0a-d4c49c6fce62 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:24.957260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 165/323] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:24.963751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bdf79-8823-4bf4-ba06-80e366dcd4f7 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:24.964114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bdf79-8823-4bf4-ba06-80e366dcd4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:24.964345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bdf79-8823-4bf4-ba06-80e366dcd4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:24.983254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d83bdf79-8823-4bf4-ba06-80e366dcd4f7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:24.984503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d83bdf79-8823-4bf4-ba06-80e366dcd4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:25.058194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 159/324] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:25.080712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd1442-ee8a-4d59-ba0a-d4c49c6fce62 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.097586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:25.200348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd1442-ee8a-4d59-ba0a-d4c49c6fce62 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.220306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d434207-c5df-4324-9894-6064ca6206d6 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.221119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d434207-c5df-4324-9894-6064ca6206d6 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.221400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d434207-c5df-4324-9894-6064ca6206d6 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.222942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 160/325] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:25.228789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.229162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.229603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.239011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.240094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.240534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.240982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.252946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:25.253300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:25.254251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:25.262751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:25.263665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:25.264474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:25.266796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 166/326] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:25.274841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.293803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:25.350066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 161/327] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:25.364158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:25.373184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.404143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645143bd-16d6-45a6-93f2-b564023cb9c9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.448763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.450113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.450870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.451620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.452837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:25.453361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:25.455107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:25.471255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 167/328] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:25.474022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.516091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ef99a-4c3e-42a4-8dc3-80faa4521240 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.559660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.561469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.561970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.562670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.579113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:25.590909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:25.591823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:25.592669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:25.594631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 162/329] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:25.599367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.613549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263545d7-100d-46d0-be7e-c7da862f6d10 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.630772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:25.652033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.653335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.653810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.654270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] PATH_INFO: `/v3/users/629c17862f6741ff84f18939fc7c2fc2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.655816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] RBAC: Authorizing `identity:get_user(user_id=629c17862f6741ff84f18939fc7c2fc2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:25.657541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:25.663977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 168/330] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/users/629c17862f6741ff84f18939fc7c2fc2 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:25.666059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.669255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:25.714671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9318686-3b06-4230-a014-360d24c5c408 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.751367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.752079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.752360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.752633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.753398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:25.753704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:25.755210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:25.764514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 163/331] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:25.770119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:25.779268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366876f-a467-48fa-a030-4715d43cc9d0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.816929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.818302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.818778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.819246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] PATH_INFO: `/v3/system/users/629c17862f6741ff84f18939fc7c2fc2/roles/34639ea93eb34162ac3fe50f87d9e581` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.821151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=629c17862f6741ff84f18939fc7c2fc2, role_id=34639ea93eb34162ac3fe50f87d9e581)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:25.822895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:25.836028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 169/332] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Jun 30 09:47:25 2022] PUT /identity/v3/system/users/629c17862f6741ff84f18939fc7c2fc2/roles/34639ea93eb34162ac3fe50f87d9e581 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:25.870435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:25.883604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdbb29-ad7d-4da8-8ce6-b6daa8bbee57 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:25.921689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e21b-cded-4d43-87e2-0714dd1d8ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:25.922598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e21b-cded-4d43-87e2-0714dd1d8ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:25.922929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e21b-cded-4d43-87e2-0714dd1d8ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:25.923207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e21b-cded-4d43-87e2-0714dd1d8ff2 None admin] PATH_INFO: `/v3/users/8030ea88088047e1b71ca27f8b47fe35` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:25.924498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e21b-cded-4d43-87e2-0714dd1d8ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=8030ea88088047e1b71ca27f8b47fe35)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:25.925972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e21b-cded-4d43-87e2-0714dd1d8ff2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:25.928082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 164/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/users/8030ea88088047e1b71ca27f8b47fe35 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:25.934359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:26.010138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:26.056128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305cdd2-7ced-423c-8d2b-2bd35bc8cd20 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:26.093581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:26.094247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:26.094531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:26.094811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:26.095657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] RBAC: Adding query filter params (role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, user.id=8030ea88088047e1b71ca27f8b47fe35, scope.system=all) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:26.096047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:26.097508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f63e74-8fc9-480d-8dab-4d673efaccf7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:26.110314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 170/334] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 09:47:25 2022] GET /identity/v3/role_assignments?user.id=8030ea88088047e1b71ca27f8b47fe35&scope.system=all&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:27.599577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4fc28-f56c-4a3c-a1fd-03e33e388a8a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:27.600121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4fc28-f56c-4a3c-a1fd-03e33e388a8a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:27.600197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4fc28-f56c-4a3c-a1fd-03e33e388a8a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:27.601802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 165/335] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:27.607019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7eff7f-c6bd-4019-878c-355620fe2a08 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:27.607484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7eff7f-c6bd-4019-878c-355620fe2a08 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:27.607557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7eff7f-c6bd-4019-878c-355620fe2a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:27.632886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9a7eff7f-c6bd-4019-878c-355620fe2a08 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:27.634142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9a7eff7f-c6bd-4019-878c-355620fe2a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:27.726401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 171/336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:27.766726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4fc28-f56c-4a3c-a1fd-03e33e388a8a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:27.778242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3084d-e46a-4130-b58e-a23c47113b52 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:27.778526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3084d-e46a-4130-b58e-a23c47113b52 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:27.778806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3084d-e46a-4130-b58e-a23c47113b52 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:27.780222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 172/337] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:27.790465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:27.790841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:27.791122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:27.831068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:27.834882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:27.928829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:27.956309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 173/338] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:27.982613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:27.988650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4fc28-f56c-4a3c-a1fd-03e33e388a8a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.048762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.049946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.050628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.052184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.072515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:28.080361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:28.081199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:28.082423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 166/339] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:28.093060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:28.115169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:28.147106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8a064-bfe0-458e-9da3-f5f1e87a9aec None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.198770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.199522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.199800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.200098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.213549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f876e-8d0e-4030-8b6a-0e8478389904 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.217243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:28.233402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:28.234330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:28.235128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 174/340] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:28.240844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:28.258376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.259393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.259941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.260408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.261402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:28.261939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:28.263513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:28.278838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 167/341] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:28.286179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:28.303547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:28.375971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b9278-3129-43e2-8386-f67547c30c66 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.397963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:28.426037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1a057-79cd-4704-be6a-94ff4f3b0896 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.432595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.433590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.434097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.434563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.435520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:28.436044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:28.437622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:28.456619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 175/342] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:28.457847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:28.471108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.471864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.472073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.472341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.546321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:28.547792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:28.596871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e535656-1a6b-4801-9a11-e2eeec3b47c0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.615895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:28.640349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:28.661581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 168/343] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:28.670740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:28.713172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.714189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.714719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.715173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.790107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de831ce-4b29-42dd-8d1c-6c6aafac3ff0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.825790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.826673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.826956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.827224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.828447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:28.830009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:28.831972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 169/344] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:28.837756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:28.889233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:28.933639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:28.936442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:28.943058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7c8ec-4d6f-4df8-a22a-b0b1025fac13 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:28.979592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:28.980426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:28.980708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:28.980978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:28.987616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:28.989141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:28.991304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 170/345] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:28.997076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:29.105311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36df13d-8e6f-46be-bf98-a3cad5d5ecdc None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:29.116275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:29.142153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:29.142920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.143204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.143480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.144371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=f525943eff304b3786a90a844c39a37f) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:29.144685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:29.146133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e644f9f-f84b-405e-b99c-c2bc5f04583f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:29.153184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 171/346] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:29.154478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c16aa-c05d-4eed-a7f6-ccd76dba2f45 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.154764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c16aa-c05d-4eed-a7f6-ccd76dba2f45 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.155040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c16aa-c05d-4eed-a7f6-ccd76dba2f45 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.160835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 172/347] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:29.167190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.167484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.167746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.222796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:29.224940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:29.244203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 176/348] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 786 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:29.251080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:29.281049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:29.488506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 173/349] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:29.530284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176be3a6-9647-4522-aca5-6ebe5087708b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:29.583076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:29.591913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:29.591913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.591913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.591913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.596670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:29.598163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:29.600340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 177/350] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:29 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:29.607944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:29.718463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:29.752095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df494ac-4d70-47e2-9f41-37703f32db6c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:29.808744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1318-34d8-4639-befd-03714a968898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:29.809502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1318-34d8-4639-befd-03714a968898 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.809944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1318-34d8-4639-befd-03714a968898 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.810378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1318-34d8-4639-befd-03714a968898 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.811764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f1318-34d8-4639-befd-03714a968898 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:29.813385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f1318-34d8-4639-befd-03714a968898 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:29.816551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 178/351] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:29 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:29.821734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1318-34d8-4639-befd-03714a968898 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:29.848695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a70986-42ec-4600-bd38-f34017aec8ba None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:29.852870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:29.907715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115cbec-0fd5-4fed-b7c0-f7365e6e2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:29.910625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115cbec-0fd5-4fed-b7c0-f7365e6e2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.911316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115cbec-0fd5-4fed-b7c0-f7365e6e2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.912023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115cbec-0fd5-4fed-b7c0-f7365e6e2fbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.930658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115cbec-0fd5-4fed-b7c0-f7365e6e2fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b115cbec-0fd5-4fed-b7c0-f7365e6e2fbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:29.945901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:29.947840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:29.949742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:29.953639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 174/352] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:29.956031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c9678-6999-43ec-826d-cd8bcdc528d5 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:29.956664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c9678-6999-43ec-826d-cd8bcdc528d5 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:29.957286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c9678-6999-43ec-826d-cd8bcdc528d5 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:29.960091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 175/353] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:29.961513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c9678-6999-43ec-826d-cd8bcdc528d5 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:30.098762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1318-34d8-4639-befd-03714a968898 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:30.131491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c9678-6999-43ec-826d-cd8bcdc528d5 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:30.186943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:30.188491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.189236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.189978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.191532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:30.192285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:30.194977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccbebd-cd52-4b68-a742-114a04aa3d24 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:30.199070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:30.200265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.200728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.201182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.202629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=f525943eff304b3786a90a844c39a37f) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:30.203140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:30.205976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:30.210399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 176/354] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:30.212635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d76c0-4f6f-4a9c-ac28-26223aeb8e61 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.213088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d76c0-4f6f-4a9c-ac28-26223aeb8e61 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.213495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d76c0-4f6f-4a9c-ac28-26223aeb8e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.217070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 179/355] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:29 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=805a0b1c9f554aff8a16769124f939eb => generated 260 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:30.221899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:30.246710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bf4d76c0-4f6f-4a9c-ac28-26223aeb8e61 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:30.249374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bf4d76c0-4f6f-4a9c-ac28-26223aeb8e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:30.252232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:30.287380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:30.343064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 177/356] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:30.344731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df9dff-6f10-455f-b6df-df510f8cd6bb None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.345334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df9dff-6f10-455f-b6df-df510f8cd6bb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.345745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df9dff-6f10-455f-b6df-df510f8cd6bb None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.347674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 178/357] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:30.357041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.357698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.358110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.382446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:30.384705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:30.463443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 179/358] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:30.465477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:30.497503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050e03c-7d1f-42d3-a68a-d2ab5a49878e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:30.556013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5439-0073-43e9-ba90-b53497011863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:30.556925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5439-0073-43e9-ba90-b53497011863 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.557363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5439-0073-43e9-ba90-b53497011863 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.557773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5439-0073-43e9-ba90-b53497011863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.562782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:30.582085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4c06e-e7bf-477b-8b36-9bc996b4590b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:30.627778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:30.629010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.629480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.629938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.642294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:30.651812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:30.652748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 09:47:30.653559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:30.654141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f5439-0073-43e9-ba90-b53497011863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:30.655148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f5439-0073-43e9-ba90-b53497011863 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:30.656578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 180/359] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:30.657684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:30.664929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:30.774119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f121-8cc7-4591-bd6e-b9eeee110ae1 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:30.789402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 180/360] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:30.790919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165a1a0-d0c4-4888-9516-dc765a5acb93 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.791196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165a1a0-d0c4-4888-9516-dc765a5acb93 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.791481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165a1a0-d0c4-4888-9516-dc765a5acb93 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.793178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 181/361] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:30.793909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165a1a0-d0c4-4888-9516-dc765a5acb93 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:30.813656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16f7d9-dba0-4436-b771-7bbabcb95b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:30.814976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f7d9-dba0-4436-b771-7bbabcb95b03 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.815472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f7d9-dba0-4436-b771-7bbabcb95b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.816013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f7d9-dba0-4436-b771-7bbabcb95b03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.827786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16f7d9-dba0-4436-b771-7bbabcb95b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-1a16f7d9-dba0-4436-b771-7bbabcb95b03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:30.837494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:30.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:30.839156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:30.841621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 181/362] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:30.843115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.843602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.844097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.856817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:30.862932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:30.864690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:30.909086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165a1a0-d0c4-4888-9516-dc765a5acb93 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:30.919281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:30.938012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 182/363] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:30.939177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:30.950950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:30.951888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:30.952138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:30.952427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:30.953199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:30.953509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:30.954903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:30.973779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 182/364] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:30.974691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.048535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125c8ef-5f95-4ac7-8d74-81b48971e0a5 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.083811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.085011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.085519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.085953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.087890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.089526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.091389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 183/365] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:30 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.092342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.096555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46466-e497-4e14-ac46-ca509e2585cb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.098044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.133085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.133779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.134064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.134336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.135095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:31.135403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.136839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.146856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 183/366] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.147738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.180853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.208878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da928810-ea6d-4525-acef-f42215900bfc None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.247138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.249341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.249948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.250409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] PATH_INFO: `/v3/users/629c17862f6741ff84f18939fc7c2fc2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.251833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] RBAC: Authorizing `identity:get_user(user_id=629c17862f6741ff84f18939fc7c2fc2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.253768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.256011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 184/367] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/users/629c17862f6741ff84f18939fc7c2fc2 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.256138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb465d-93f2-4773-8847-36a9ca5d0004 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.257183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.300297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.301111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.301391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.301740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.312821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:31.320738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:31.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:31.322415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:31.323018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 184/368] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:31.324186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.371690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2532ea9-9fb9-4e76-837a-6af0e3ea7eef None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.385645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.410148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.411502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.412041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.412641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.414000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.415945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.418076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 185/369] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.419059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.431582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.440840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19474a-5c7b-4beb-bbd7-25063f4579fa None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.478976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.479639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.479978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.480264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.530495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed0edf-ab03-4a67-a61a-f06cd8b1d726 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.551530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.553152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.569372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.570035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.570313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.570583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.571542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] RBAC: Adding query filter params (role.id=34639ea93eb34162ac3fe50f87d9e581, user.id=629c17862f6741ff84f18939fc7c2fc2, scope.system=all) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:31.571852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.573356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.579443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.590899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 186/370] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/role_assignments?user.id=629c17862f6741ff84f18939fc7c2fc2&scope.system=all&role.id=34639ea93eb34162ac3fe50f87d9e581 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.591829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.656274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.663383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 185/371] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:31.669334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.761032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efa4a1-de79-4be5-a726-457cf280b802 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.788745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab5e30-6d84-4402-9315-da8d2bd28797 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.810646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.811434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.811720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.812025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.812795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:31.813104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.814611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.830268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 187/372] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.831480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.833258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:31.833932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.834212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.834490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.835523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=0760678cdad243f09545e4bf1c3266ba) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:31.835842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:31.837773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4229f13-8e51-468f-b5c9-b53441d4b936 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:31.844979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 186/373] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:31.846443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73427d4-f789-48a3-ad0d-ab285824d95d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:31.846736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73427d4-f789-48a3-ad0d-ab285824d95d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:31.847014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73427d4-f789-48a3-ad0d-ab285824d95d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:31.848335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 187/374] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:31.849068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73427d4-f789-48a3-ad0d-ab285824d95d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:31.918772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.935098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:31.970245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73427d4-f789-48a3-ad0d-ab285824d95d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:31.984515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405143cc-a563-416a-956f-454c3eb60cb9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.007696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.008456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.008738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.009010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.028032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d6f64-da8b-4045-93df-0d05611d527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.029015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d6f64-da8b-4045-93df-0d05611d527f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.029351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d6f64-da8b-4045-93df-0d05611d527f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.029770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d6f64-da8b-4045-93df-0d05611d527f None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.031320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d6f64-da8b-4045-93df-0d05611d527f None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.033025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d6f64-da8b-4045-93df-0d05611d527f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.035350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 188/375] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:32.036791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.037079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.037365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.063029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:32.064277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:32.089660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.091415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.152584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:32.165299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 189/376] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:32.166907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.222772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 188/377] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:32.223796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.251703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:32.299827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b982a4-b7a8-4008-9edc-d2c4384a6709 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.331767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba6acc-4046-413f-b375-668f2ba0486e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.337678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.338569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.338848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.339120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.340367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.341968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.345183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:32.345985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.379176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.380176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.380461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.381043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.396831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:32.407117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:32.408151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:32.408987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 189/379] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:32.410242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.434198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:32.477320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a382c9b-9966-433d-9b97-f81e1d986929 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.509266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:32.520718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.521647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.522034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.522360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] PATH_INFO: `/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.523788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] RBAC: Authorizing `identity:get_project(project_id=e04fd91b7cf34c948c016c3c7a5bd4df)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.528050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.530340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 191/380] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:32.531187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.556247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8548138-6e95-4988-b316-010ba229a9c2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.604626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.605329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.605988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.606278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.607380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=0760678cdad243f09545e4bf1c3266ba) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:32.607700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.609174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.617807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 190/381] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=805a0b1c9f554aff8a16769124f939eb => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:32.618952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.692274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac146-e5bd-4ba1-9883-fb71133c8701 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.721379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:32.751379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c63eac-878a-43c5-ac5e-e57a02fa4e47 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:32.775247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.776130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.776643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.777136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.777988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:32.778505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.780174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d27721-5a61-4fe1-a0b7-91dfed374471 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.793742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 192/382] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:32.794370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819ef3d-0b13-4a7b-9201-a5e58ecca291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:32.795370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab42d-f42d-46fc-9a95-9e58084f6f40 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.795628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ef3d-0b13-4a7b-9201-a5e58ecca291 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.795835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab42d-f42d-46fc-9a95-9e58084f6f40 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.796225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ef3d-0b13-4a7b-9201-a5e58ecca291 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.796668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ef3d-0b13-4a7b-9201-a5e58ecca291 None admin] PATH_INFO: `/v3/groups/190ecbc0963246b1abf2aef301f8bc8a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.800253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab42d-f42d-46fc-9a95-9e58084f6f40 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.802175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 193/383] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:32.804589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdab42d-f42d-46fc-9a95-9e58084f6f40 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.812127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819ef3d-0b13-4a7b-9201-a5e58ecca291 None admin] RBAC: Authorizing `identity:get_group(group_id=190ecbc0963246b1abf2aef301f8bc8a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:32.813829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819ef3d-0b13-4a7b-9201-a5e58ecca291 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:32.817399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 191/384] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/groups/190ecbc0963246b1abf2aef301f8bc8a => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:32.818951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:32.819379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:32.819787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:32.843819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:32.845095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:32.890263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:32.928950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 192/385] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:32.930012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:32.953501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdab42d-f42d-46fc-9a95-9e58084f6f40 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.009369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.010297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.012492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.012815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.014082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.014369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:33.015547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.017768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 194/386] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:33.018832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:33.051404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43a957-3ea7-4399-8b1a-29a1bcb93453 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.100603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.101607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.102214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.102664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.103681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] RBAC: Adding query filter params (group.id=190ecbc0963246b1abf2aef301f8bc8a, role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=e04fd91b7cf34c948c016c3c7a5bd4df) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:33.104184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.105818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.113070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 193/387] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:32 2022] GET /identity/v3/role_assignments?group.id=190ecbc0963246b1abf2aef301f8bc8a&scope.project.id=e04fd91b7cf34c948c016c3c7a5bd4df&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:33.113886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:33.182382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd39e0a-39d1-4024-b1e7-87aa821bfb3c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.233591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d0b6-e471-4621-8b9a-3e6d9d9a2b8b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.240703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:33.269250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.270092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.270368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.270642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.282098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.283425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.283927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.284364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.287744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:33.288749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.290836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.293196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 194/388] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:33 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:33.296908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:33.298158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:33.298942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:33.299810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 195/389] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:33.300546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:33.307531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:33.312297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:33.454788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cadd5-0971-4c3b-9d9b-bab7c157fe68 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.464560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c136-6cc7-4469-8f4f-4d5b358a778d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.495238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.496565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.496987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.497269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f/groups/ceaa049cca6449178f699b4e56a979d2/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.502238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] RBAC: Authorizing `identity:create_grant(project_id=f525943eff304b3786a90a844c39a37f, group_id=ceaa049cca6449178f699b4e56a979d2, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.503714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.517148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.518183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.518939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.519462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.520669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:33.521290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.522961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.524703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 195/390] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 09:47:33 2022] PUT /identity/v3/projects/f525943eff304b3786a90a844c39a37f/groups/ceaa049cca6449178f699b4e56a979d2/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:33.533919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 196/391] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:33.539985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:33.614185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:33.674341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d718-f909-4e3a-ad62-150eb2ed431c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.723348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.724154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.724435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.724709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.725882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.727482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.729152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 196/392] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:33 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:33.734857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:33.796178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:33.860289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d56997-c61f-4115-81a3-1af9743ef6eb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:33.900778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7315ea-e904-448f-939e-a45fbbc9d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:33.901660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7315ea-e904-448f-939e-a45fbbc9d48e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:33.901941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7315ea-e904-448f-939e-a45fbbc9d48e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:33.902213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7315ea-e904-448f-939e-a45fbbc9d48e None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:33.908646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7315ea-e904-448f-939e-a45fbbc9d48e None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:33.910105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7315ea-e904-448f-939e-a45fbbc9d48e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:33.912158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 197/393] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:33 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:33.918349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:34.046473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d0bc0-ac7c-4cd9-9e46-b94a140e68e6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:34.082356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:34.088117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d474ea-0fae-4ae9-a402-8145aae3ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:34.089037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d474ea-0fae-4ae9-a402-8145aae3ed59 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:34.089747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d474ea-0fae-4ae9-a402-8145aae3ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:34.090286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d474ea-0fae-4ae9-a402-8145aae3ed59 None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f/groups/ceaa049cca6449178f699b4e56a979d2/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:34.093131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d474ea-0fae-4ae9-a402-8145aae3ed59 None admin] RBAC: Authorizing `identity:create_grant(project_id=f525943eff304b3786a90a844c39a37f, group_id=ceaa049cca6449178f699b4e56a979d2, role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:34.094607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d474ea-0fae-4ae9-a402-8145aae3ed59 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:34.109905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 197/394] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 09:47:33 2022] PUT /identity/v3/projects/f525943eff304b3786a90a844c39a37f/groups/ceaa049cca6449178f699b4e56a979d2/roles/805a0b1c9f554aff8a16769124f939eb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:34.416247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88dda-9139-49fd-9a8f-5e88ecb4dd8d None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:34.416776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88dda-9139-49fd-9a8f-5e88ecb4dd8d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:34.416776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d88dda-9139-49fd-9a8f-5e88ecb4dd8d None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:34.418403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 198/395] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:34.424514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:34.424803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:34.425090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:34.442737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:34.443991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:34.523194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 198/396] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:34.545988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d88dda-9139-49fd-9a8f-5e88ecb4dd8d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:34.668124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d88dda-9139-49fd-9a8f-5e88ecb4dd8d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:34.682255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:34.707092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:34.707911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:34.708195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:34.708467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:34.720249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:34.727919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:34.728800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:34.729567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:34.735129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:34.737196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 199/397] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:34.860393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebc77a-dd41-4ffc-8ede-2b3cc1bbf267 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:34.891418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:34.907133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:34.908265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:34.909708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:34.909708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:34.909708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:34.909912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:34.911214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:34.923300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 199/398] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:34.928952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:35.035921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10f4f-f77d-4634-bc25-14168a1c2e61 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:35.080687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:35.081559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.081834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.082107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.088603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:35.090080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:35.091725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 200/399] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:34 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:35.097289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:35.257302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d161a7-eb23-4e8d-b303-bfead983a775 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:35.314602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d9ec5-01bc-47b7-a7fc-632c0aadc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:35.315482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d9ec5-01bc-47b7-a7fc-632c0aadc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.315760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d9ec5-01bc-47b7-a7fc-632c0aadc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.316057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d9ec5-01bc-47b7-a7fc-632c0aadc8d2 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.317262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d9ec5-01bc-47b7-a7fc-632c0aadc8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:35.318708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d9ec5-01bc-47b7-a7fc-632c0aadc8d2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:35.324866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 200/400] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:35 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:35.330293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:35.408297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:35.435923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f2b7-1c3b-4584-b649-a94a02812821 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.436239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f2b7-1c3b-4584-b649-a94a02812821 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.436491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f2b7-1c3b-4584-b649-a94a02812821 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.438010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 201/401] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:35.443247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.443533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.443810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.469625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf3a1-0e0a-4c60-ba3c-5dd046e9671f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:35.477070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:35.478284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:35.506135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:35.507155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.507551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.507768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba/groups/ceaa049cca6449178f699b4e56a979d2/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.509958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] RBAC: Authorizing `identity:create_grant(project_id=0760678cdad243f09545e4bf1c3266ba, group_id=ceaa049cca6449178f699b4e56a979d2, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:35.511496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:35.538391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 201/402] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 09:47:35 2022] PUT /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba/groups/ceaa049cca6449178f699b4e56a979d2/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:35.546910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:35.568424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 202/403] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:35.595350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:35.728260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2628cf-1ef5-4b23-a3d3-c9d0b3652e3b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:35.729858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:35.772865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f50438-7d0a-48b7-b172-76b5a6a0a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:35.773609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f50438-7d0a-48b7-b172-76b5a6a0a3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.774033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f50438-7d0a-48b7-b172-76b5a6a0a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.774357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f50438-7d0a-48b7-b172-76b5a6a0a3b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.786048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f50438-7d0a-48b7-b172-76b5a6a0a3b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-40f50438-7d0a-48b7-b172-76b5a6a0a3b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:35.794454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:35.795413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:35.796218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 202/404] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:35.802744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:35.906165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2866-ab5a-4492-85c3-a7329f1bd021 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.906512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2866-ab5a-4492-85c3-a7329f1bd021 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.906723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2866-ab5a-4492-85c3-a7329f1bd021 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.908300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 203/405] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:35.913958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.914245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.914880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.923263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c3019-ff0d-4a36-8036-c443b663628e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:35.955345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:35.956872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:35.972311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:35.972311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:35.972311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:35.972990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:35.973343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:35.973651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:35.975052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:35.984362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 203/406] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:35.989662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.040719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:36.058947 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 204/407] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:36.094672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.107897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365d55-3e44-4ba1-93e2-43b78b599edc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.147906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.147906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.147906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.147906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.148383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:36.148383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:36.148383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 204/408] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:35 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:36.153316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.254052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:36.267480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741f64e-47e6-4e00-a982-71eee5e94807 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.312625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.314287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.314568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.314839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.326459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:36.335263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:36.336233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:36.337434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:36.338036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 205/409] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:36.344331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.370273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d406-2366-4eb7-94c6-04fdeb6560c2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.384182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:36.411592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ca676-2ac6-4736-8de3-ddc1865b34b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.412493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ca676-2ac6-4736-8de3-ddc1865b34b6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.412770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ca676-2ac6-4736-8de3-ddc1865b34b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.413012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ca676-2ac6-4736-8de3-ddc1865b34b6 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.415427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ca676-2ac6-4736-8de3-ddc1865b34b6 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:36.416911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ca676-2ac6-4736-8de3-ddc1865b34b6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:36.418842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 205/410] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:36.419863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea182f-4bdb-4dab-b82d-1a95324f70d6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.420179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea182f-4bdb-4dab-b82d-1a95324f70d6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.420457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea182f-4bdb-4dab-b82d-1a95324f70d6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.425761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 206/411] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:36.425761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea182f-4bdb-4dab-b82d-1a95324f70d6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.476405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c832c-0b4b-407b-a17b-aef38019f45f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.526266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.526951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.527235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.527514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.528338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:36.528647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:36.530566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942f787-fad3-4877-88ae-7eb3edf7ea88 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:36.542661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 206/412] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:36.543489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea182f-4bdb-4dab-b82d-1a95324f70d6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.543968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.544259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.544534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.564818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:36.566427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:36.577410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:36.592544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.593710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.593996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.594270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba/groups/ceaa049cca6449178f699b4e56a979d2/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.597201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0760678cdad243f09545e4bf1c3266ba, group_id=ceaa049cca6449178f699b4e56a979d2, role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:36.599190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:36.620882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 207/413] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 09:47:36 2022] PUT /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba/groups/ceaa049cca6449178f699b4e56a979d2/roles/805a0b1c9f554aff8a16769124f939eb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:36.621726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.673872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 207/414] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:36.701305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.702056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:36.831030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390e4e-b380-4f22-80d5-d1d26ab869a0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.867772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.869048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.869531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.870000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] PATH_INFO: `/v3/groups/190ecbc0963246b1abf2aef301f8bc8a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.871416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] RBAC: Authorizing `identity:get_group(group_id=190ecbc0963246b1abf2aef301f8bc8a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:36.873222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:36.876128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 208/415] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/groups/190ecbc0963246b1abf2aef301f8bc8a => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:36.876671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b3a49-249d-4f52-a479-41101f1ea2ac None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:36.884958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.918922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:36.920117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:36.920615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:36.920979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:36.933313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:36.942948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:36.944851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:36.946443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 208/416] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:36.950226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:36.955138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.019722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429aa1e-a433-4262-b823-ac5474d89c61 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.038596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.073168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba0070-dda4-4fd9-9b4c-9b8e440d03ae None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.080705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.081940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.082419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.082884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] PATH_INFO: `/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.084315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=e04fd91b7cf34c948c016c3c7a5bd4df)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.085994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.088657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 209/417] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:37.098633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.149520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.150200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.150550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.150816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.151587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] RBAC: Adding query filter params (name=member) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:37.151901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.154133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9a2e-1ed1-40cd-8e3e-755504b6b5da None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.166826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 209/418] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:37.168243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5f19b-6795-4744-91fd-c1bba0ae4e1b None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.168526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5f19b-6795-4744-91fd-c1bba0ae4e1b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.168799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5f19b-6795-4744-91fd-c1bba0ae4e1b None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.170274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 210/419] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:37.172025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5f19b-6795-4744-91fd-c1bba0ae4e1b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.238669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.245778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c6f22-4af1-49f3-882d-640d19d1b9f0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.275449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5f19b-6795-4744-91fd-c1bba0ae4e1b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.287912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04af7c4-03dc-470d-92c1-e54fe1f31966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.288333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af7c4-03dc-470d-92c1-e54fe1f31966 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.288920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af7c4-03dc-470d-92c1-e54fe1f31966 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.289420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04af7c4-03dc-470d-92c1-e54fe1f31966 None admin] PATH_INFO: `/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df/groups/190ecbc0963246b1abf2aef301f8bc8a/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.294369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04af7c4-03dc-470d-92c1-e54fe1f31966 None admin] RBAC: Authorizing `identity:create_grant(project_id=e04fd91b7cf34c948c016c3c7a5bd4df, group_id=190ecbc0963246b1abf2aef301f8bc8a, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.296153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04af7c4-03dc-470d-92c1-e54fe1f31966 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.310511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.311148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.311471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.311750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.314170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 210/420] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 09:47:37 2022] PUT /identity/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df/groups/190ecbc0963246b1abf2aef301f8bc8a/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:37.315937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.316414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.316922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.335649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:37.337677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:37.372762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.412973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 211/421] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:37.440535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.442370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.451555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.565303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.570191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62bcfc-b4b2-4768-b045-4ebc0da54320 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.606377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 211/422] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:37.612423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.612950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.613984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.614516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.615069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.626807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:37.635929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:37.636845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:37.637635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:37.640533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 212/423] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:37.648907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.696860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.727616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea753a6-ae78-494c-9484-cc963e695805 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.766319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.767558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.768040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.768444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.769868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.771430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.773926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 212/424] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:37.778863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.783419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb38d26-f82e-49ae-8ca5-edc335823bc0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.815963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.817631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.818756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.818756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.819125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.820017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:37.820452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.822032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.835343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 213/425] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:37.837188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.887761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb51514-78ff-4d7d-8efe-30c84d10bd1c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:37.941017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:37.942360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:37.944085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:37.944713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:37.947040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:37.949460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:37.953454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 213/426] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:37.957649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:37.959125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:37.987727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b91b92-a8b2-4428-85d2-b871ccbd5d3e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.046071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb2938-6a66-4404-b856-1417d1280c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.046748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2938-6a66-4404-b856-1417d1280c92 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.047101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2938-6a66-4404-b856-1417d1280c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.047445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2938-6a66-4404-b856-1417d1280c92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.101057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e854b1d-3ea9-4137-8f01-9ca5b742f4b7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.104350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:38.120477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb2938-6a66-4404-b856-1417d1280c92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:38.122093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb2938-6a66-4404-b856-1417d1280c92 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:38.143094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.144374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.144825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.145258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.146283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=f525943eff304b3786a90a844c39a37f) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:38.146760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:38.148631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:38.156289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 214/427] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:38.215611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:38.233527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 214/428] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:38.235067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:38.239595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d232719-1199-4afa-bc0c-c9a3f4d6bcd9 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.239903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d232719-1199-4afa-bc0c-c9a3f4d6bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.240188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d232719-1199-4afa-bc0c-c9a3f4d6bcd9 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.242155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 215/429] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:38.253702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.254009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.254746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.285632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:38.288261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:38.368671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 216/430] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:38.393463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:38.447303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae234-5da5-4092-8231-5d3080c80d87 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.494084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.495370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.495863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.496356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] PATH_INFO: `/v3/projects/f525943eff304b3786a90a844c39a37f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.500634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] RBAC: Authorizing `identity:get_project(project_id=f525943eff304b3786a90a844c39a37f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:38.502396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:38.504382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 215/431] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:38 2022] GET /identity/v3/projects/f525943eff304b3786a90a844c39a37f => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:38.514526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:38.524925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e4035-3203-497d-bd8b-04400e375960 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.547278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:38.555830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:38.567195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.567912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.568237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.568404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.579406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:38.589420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:38.590308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:38.591113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 217/432] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:38.597151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:38.638533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4dc9f-ddff-4d63-a43c-b97278a9120f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.686842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.687824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.688236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.688603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.690135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:38.691764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:38.694645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 216/433] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:38 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:38.700842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:38.736195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619503e-a846-4c9a-9f6c-fb1e69a6eaf7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.778567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.779711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.780224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.780688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.781825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:38.782347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:38.783957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:38.795626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 218/434] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:47:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:38.800788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:38.825200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31469-59fb-4bde-992a-9e24df9c49d7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.826808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:38.855530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:38.873782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.874886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.875385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.875981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:38.877156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=f525943eff304b3786a90a844c39a37f) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:38.877725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:38.879599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8d16f-cc7a-4904-8f4a-6a9770c701fb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:38.887430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 217/435] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:38 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=f525943eff304b3786a90a844c39a37f&role.id=805a0b1c9f554aff8a16769124f939eb => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:38.927424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35aa76e-5c12-4601-bdc6-4869d72304b8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:38.968062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:38.968651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:38.968935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:38.969216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.044586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:39.046077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:39.088753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:39.122934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b424523-e561-4fe3-a322-f2a85cfecd51 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.123245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b424523-e561-4fe3-a322-f2a85cfecd51 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.123514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b424523-e561-4fe3-a322-f2a85cfecd51 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.125203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 218/436] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:39.132220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.132506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.132781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.158047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 219/437] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 09:47:39.158456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:39.159748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:39.164844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:39.245506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 219/438] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:39.269142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:39.285488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c6bef-d004-4808-a2b7-3457260c870d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:39.326959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:39.327843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.328152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.328437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.329729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:39.331194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:39.334144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 220/439] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:39.341315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:39.356062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:39.401654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:39.409490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11597d-74e1-46bc-817a-f8fa75987c18 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:39.454274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:39.455149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.455419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.455685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.466910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:39.473547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e625f6a-0202-490c-97c9-769502d7d95c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:39.475682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:39.476833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:39.477914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 220/440] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:39.482679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:39.513156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:39.513937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.514219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.514501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.515679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:39.517221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:39.518857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 221/441] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:39.524576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:39.590891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a368a9a-6ba0-423b-a652-0f4356ac96a0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:39.603951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:39.630360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cbfc4-eeca-4143-8f85-3261a3d38389 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:39.633791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:39.634612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.634983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.635343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.636232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:39.636634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:39.638114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:39.647736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 221/442] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:39.653370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:39.669207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:39.669887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.670157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.670430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.671260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=b16e6fc6c0ee491e8078bc37c8c12d0c, scope.project.id=0760678cdad243f09545e4bf1c3266ba) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:39.671567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:39.673065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afb1a7-869e-4f2e-a91a-d28c859d2e37 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:39.673444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:39.687913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 222/443] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=b16e6fc6c0ee491e8078bc37c8c12d0c => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:39.780752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939beb09-ce1b-4d9f-8700-1e7a7e2b7589 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:39.818907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:39.819707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.820120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.820462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.857669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebe33c-167f-4e1f-b79e-666db10c40f7 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.858212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebe33c-167f-4e1f-b79e-666db10c40f7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.858633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebe33c-167f-4e1f-b79e-666db10c40f7 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.860170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 223/444] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:39.865383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:39.865888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:39.866372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:39.885219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:39.886399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:39.897404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:39.899248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:39.920950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:39.965755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 224/445] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:39.988923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.013011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 222/446] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:40.018898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.074627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.105163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda74a8d-2a02-463b-9592-a37ded6c730f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.121619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9c7fb-fbdf-476c-9cf2-a4d8a42b2de3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.137711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.137998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.138751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.138751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.139033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.149421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:40.156191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:40.156831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:40.157810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:47:40.162682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:40.163318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.163318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.163318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] PATH_INFO: `/v3/projects/0760678cdad243f09545e4bf1c3266ba` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.163492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 225/447] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:40.163558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=0760678cdad243f09545e4bf1c3266ba)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.163558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.163558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 223/448] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/projects/0760678cdad243f09545e4bf1c3266ba => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:40.163999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.166971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.262759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b1259-62a4-4422-ac70-bf90207a5288 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.266630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132a05-5a48-498a-a476-627052da4ed8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.294173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.300374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.300606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.301046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.301323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.301504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.301634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.301735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.302028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] PATH_INFO: `/v3/groups/ceaa049cca6449178f699b4e56a979d2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.302372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:40.302672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.303209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] RBAC: Authorizing `identity:get_group(group_id=ceaa049cca6449178f699b4e56a979d2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.304074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.304681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.306382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 224/449] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/groups/ceaa049cca6449178f699b4e56a979d2 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:40.312270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.312738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 226/450] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:40.318017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.366955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.410361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f2ef6-738a-4ae8-8bfc-3c8643d08ea2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.415225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021247a0-1bb6-4df8-a67e-a5689167f8ab None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.444672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.445327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.445826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.446107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.446942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] RBAC: Adding query filter params (group.id=ceaa049cca6449178f699b4e56a979d2, role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=0760678cdad243f09545e4bf1c3266ba) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:40.447257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.447939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.448588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.448732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.448868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.449144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.454756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 225/451] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/role_assignments?group.id=ceaa049cca6449178f699b4e56a979d2&scope.project.id=0760678cdad243f09545e4bf1c3266ba&role.id=805a0b1c9f554aff8a16769124f939eb => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:40.512401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.523408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.525022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.631432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 227/452] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 09:47:40.637241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.740851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615a445-7a19-41d7-bdff-ac6b609d4d2b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.758377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.775384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.776326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.776585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.776857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] PATH_INFO: `/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.778030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e04fd91b7cf34c948c016c3c7a5bd4df)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.779477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.781209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 226/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/projects/e04fd91b7cf34c948c016c3c7a5bd4df => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:40.786951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:40.878092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:40.891182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387899-29f2-46dc-9efb-0d01f178df79 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:40.926413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887ac6-97f4-464c-a678-f6bdc09133ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:40.927265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887ac6-97f4-464c-a678-f6bdc09133ce None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:40.927545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887ac6-97f4-464c-a678-f6bdc09133ce None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:40.927821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887ac6-97f4-464c-a678-f6bdc09133ce None admin] PATH_INFO: `/v3/groups/190ecbc0963246b1abf2aef301f8bc8a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:40.929156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887ac6-97f4-464c-a678-f6bdc09133ce None admin] RBAC: Authorizing `identity:get_group(group_id=190ecbc0963246b1abf2aef301f8bc8a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:40.930714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887ac6-97f4-464c-a678-f6bdc09133ce None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:40.932368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 228/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/groups/190ecbc0963246b1abf2aef301f8bc8a => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:40.939173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:41.053350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0adc29-1186-4444-9c88-88f6263ad0e9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:41.093960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:41.094607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:41.094887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:41.095156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:41.096016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] RBAC: Adding query filter params (group.id=190ecbc0963246b1abf2aef301f8bc8a, role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=e04fd91b7cf34c948c016c3c7a5bd4df) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:41.096331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:41.097793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4053823-b99f-488b-b782-cb41bcd7b9e3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:41.103887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 227/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 09:47:40 2022] GET /identity/v3/role_assignments?group.id=190ecbc0963246b1abf2aef301f8bc8a&scope.project.id=e04fd91b7cf34c948c016c3c7a5bd4df&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:43.465440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5290d-3bc9-4c71-810b-cd62f9b588cf None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.466306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5290d-3bc9-4c71-810b-cd62f9b588cf None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.466718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5290d-3bc9-4c71-810b-cd62f9b588cf None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.468556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 229/456] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:43.473562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.474331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.474751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.504886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:43.506334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:43.566592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:43.585254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1406b60-da53-4c6a-883a-edf589c1b457 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.585895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1406b60-da53-4c6a-883a-edf589c1b457 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.587028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1406b60-da53-4c6a-883a-edf589c1b457 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.591968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 230/457] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:43.594222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.594477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 228/458] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:43.594973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.595324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.624897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:43.626095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:43.640737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:43.763487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 231/459] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:43.801131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fe89e-6dbc-4a7f-b198-ea8f96234b52 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:43.804066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:43.850544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:43.851900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.852531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.853014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.864214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:43.866033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-18848acc-7b09-45da-8ad7-7365faa5456b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:43.885001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:43.885890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:43.886744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:43.886744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:43.886744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:43.886744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:43.889101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 229/460] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:43.890688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40de22-eb43-48b2-878e-ceb8afd30fc8 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.891224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40de22-eb43-48b2-878e-ceb8afd30fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.891712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40de22-eb43-48b2-878e-ceb8afd30fc8 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.897323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 230/461] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:43.898365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40de22-eb43-48b2-878e-ceb8afd30fc8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:43.910690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:43.921848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4b362-118a-46f5-94b2-7e65c3268377 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:43.935302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:43.963962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:43.965036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:43.965322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:43.965596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:43.979522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:43.981260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-611d7021-03dd-4a5a-9866-c1379b112625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:44.002987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:44.004179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:44.005049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:44.005049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:44.005049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:44.005049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:44.011743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 232/462] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:44.013361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.013846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.014319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.017446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40de22-eb43-48b2-878e-ceb8afd30fc8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.032768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:44.034866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:44.061601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.062274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.062708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.062812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.063531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:44.063837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.065273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.077657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 231/463] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:44.078499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.123999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 233/464] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:44.123999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.204213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:44.213531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ad987-034e-459d-8b4c-3efdedf47bdd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.217698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:44.248802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.249707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.250122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.250616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.251561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:44.252097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.253292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bd6a2-7967-4dc1-8367-07a2e136ef31 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.253961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.266932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 232/465] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:44.268103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.299602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.300625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.301041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.301481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.302340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.304170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.331151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 234/466] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:44 2022] POST /identity/v3/users => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:44.332445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.411339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b1de5-7061-4711-a655-c6450fb71101 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.447408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.448327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.448544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.448813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.453475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61f68c-2cd5-4f69-bd73-4432c141de73 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.460048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.462242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ca6295ba-4613-4141-a47a-7d160f73f0c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:44.482394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:44.483955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:44.485014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:44.485014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:44.485014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:44.485014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:44.488671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 233/467] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:44.489586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:44.490145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614f09-e55d-4866-9bbf-55ee6f39904b None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.490428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614f09-e55d-4866-9bbf-55ee6f39904b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.490989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614f09-e55d-4866-9bbf-55ee6f39904b None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.496702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 234/468] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:44.498444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d614f09-e55d-4866-9bbf-55ee6f39904b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.501957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cfadf-e201-4a6b-a8b1-8c69fb5ba6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.502717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:44.504052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cfadf-e201-4a6b-a8b1-8c69fb5ba6cc None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.504668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cfadf-e201-4a6b-a8b1-8c69fb5ba6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.505164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cfadf-e201-4a6b-a8b1-8c69fb5ba6cc None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.506156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cfadf-e201-4a6b-a8b1-8c69fb5ba6cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.507844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cfadf-e201-4a6b-a8b1-8c69fb5ba6cc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.541695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 235/469] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:44 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:44.541695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93795e2e-d944-436b-9892-c2247c51937d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.541975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93795e2e-d944-436b-9892-c2247c51937d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.542308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93795e2e-d944-436b-9892-c2247c51937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.565650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-93795e2e-d944-436b-9892-c2247c51937d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:44.566919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-93795e2e-d944-436b-9892-c2247c51937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:44.644060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d614f09-e55d-4866-9bbf-55ee6f39904b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.650183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 236/470] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:44.682286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93795e2e-d944-436b-9892-c2247c51937d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.689715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.690733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.691216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.691722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.692662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:44.693160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.694747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.708808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 235/471] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:44.715846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:44.797414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:44.829005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93795e2e-d944-436b-9892-c2247c51937d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.842787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb02bc-a938-4b85-bdac-b5a30fb0b2c8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:44.850030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:44.878767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.880161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.881000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.881668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.885390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:44.886136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:44.886497 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:44.886829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:44.887603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.889172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.894174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:44.895852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-f05f09f0-3cf9-4592-baa0-cb3498f76a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:44.916854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:44.917763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:44.918601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:44.918601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:44.918601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:44.918601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:44.918829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 236/472] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:44 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:44.920987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 237/473] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:44.927466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:45.050923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c896d-3962-402d-a0de-396c1564ae6a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:45.079788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:45.099734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540764ef-ff99-47ea-9d7f-cce1e78db006 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.100299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540764ef-ff99-47ea-9d7f-cce1e78db006 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.100710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540764ef-ff99-47ea-9d7f-cce1e78db006 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.102294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 238/474] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:45.106624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:45.107624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.108108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.108548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.108814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.109107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.109389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.110024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:45.110496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:45.112097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:45.126697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 237/475] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:45.132225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:45.153074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:45.154304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:45.257223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 239/476] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:45.278221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924f931-aecd-46cd-812e-736566b56277 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:45.280736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:45.330145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:45.331836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.332307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.332768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.333650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:45.336283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:45.364336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 238/477] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:45 2022] POST /identity/v3/users => generated 249 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:45.403004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:45.410367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822541d-2c93-491e-a915-05cdfd9a737e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:45.458970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:45.459673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.460078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.460361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.461239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:45.463692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:45.473155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.sql.core [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 09:47:45.473155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 09:47:45.473155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [parameters: {'id': 'ef47077346364ce695e80cf44059eac7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 09:47:45.473155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80150) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Jun 30 09:47:45.491445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 09:47:45.492924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application yield resource Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise self.value Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application session.commit() Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 09:47:45.493815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.flush() Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application compat.raise_( Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 09:47:45.494667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 09:47:45.496979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 09:47:45.498896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [parameters: {'id': 'ef47077346364ce695e80cf44059eac7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:45.500873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 09:47:45.502731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:45.504750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 240/478] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 09:47:45 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:45.504880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:45.542181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:45.616547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375fe6-4a0b-4bee-a2f3-c7bc90c72f59 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:45.653175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91735f-58f4-4c08-b0e5-6f8247395a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:45.653997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91735f-58f4-4c08-b0e5-6f8247395a7e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.654286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91735f-58f4-4c08-b0e5-6f8247395a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.654557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91735f-58f4-4c08-b0e5-6f8247395a7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.665859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91735f-58f4-4c08-b0e5-6f8247395a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-9b91735f-58f4-4c08-b0e5-6f8247395a7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:45.673953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:45.676048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:45.677990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 239/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:47:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:45.685290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:45.797870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477b5aa-9474-44fd-8f7d-eea44413325e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:45.834844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:45.835524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:45.835809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:45.836145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:45.837986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:45.838299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:45.839698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb4abb-6327-4ea4-8ed9-bf1ea9f13cb6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:45.848867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 241/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:46.989996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b220484-4088-4414-93ea-3211b7007c37 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:46.991039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b220484-4088-4414-93ea-3211b7007c37 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:46.991513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b220484-4088-4414-93ea-3211b7007c37 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:46.993437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 240/481] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:46.998650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:46.999456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:46.999984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.020697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:47.022203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:47.043923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:47.050984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362eda6-ccc2-456d-a934-f4cc1d75d8c7 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.051270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362eda6-ccc2-456d-a934-f4cc1d75d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.051567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362eda6-ccc2-456d-a934-f4cc1d75d8c7 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.053267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 241/482] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:47.058344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.058670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.058894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.090513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:47.093612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:47.144739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 242/483] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:47.196743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.199583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 242/484] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:47.236948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.306157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc4a7e-1dbc-4aa3-97e8-7afdd5e8b83b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:47.308264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:47.341426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:47.342230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.342504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.342773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.353522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:47.361106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:47.362026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:47.362817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 243/485] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:47.370548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.401668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:47.425693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56cad3-417b-469f-94e9-1dc356738659 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:47.466376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:47.467850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.468293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.468697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.481727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:47.491212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:47.492439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:47.493563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:47.496008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 243/486] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:47.512987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.543862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe86c1-b764-4514-9165-bab0a573e862 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:47.584715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:47.585829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.585829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.585829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.587171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:47.587478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:47.588901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:47.607561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 244/487] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:47.611113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.682959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:47.707641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30faad-6c9a-4ba2-82bc-76d132620625 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:47.719792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:47.748329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:47.749118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.749464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.749805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.750443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fe854-24c7-456a-a581-4e13fe538f0a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:47.750682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:47.751051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:47.752544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd63395-e172-4bd4-a59c-0515969e8dc6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:47.761724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 244/488] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:47.763106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285aa46-ea70-40bd-bb3c-0a180ac72ece None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.763571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285aa46-ea70-40bd-bb3c-0a180ac72ece None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.763857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285aa46-ea70-40bd-bb3c-0a180ac72ece None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.765288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 245/489] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:47.768939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285aa46-ea70-40bd-bb3c-0a180ac72ece None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.793942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:47.795241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.796083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.796634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.811127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:47.813326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-77731d9a-8776-47a9-bb8a-b06e9653dbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:47.838285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:47.839357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:47.840339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:47.840339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:47.840339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:47.840339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:47.841201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 245/490] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:47.842405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.842691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.842966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.860021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:47.861250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:47.892980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285aa46-ea70-40bd-bb3c-0a180ac72ece None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:47.937115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:47.938201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:47.938633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:47.939057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:47.940230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 246/491] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:47.941258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:47.952862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:47.955091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:47.965541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:47.978785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:47.979673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:47.980573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:47.980573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:47.980573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:47.980573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:47.983371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 246/492] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:47.984456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.022095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.069151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6ac9-2142-4f22-a5bf-b74e3c2fbb61 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.105721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.106409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.106751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.107117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.107951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:48.108327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.109791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.119616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db01a2-c5f5-4bcc-972b-7d28c5550c76 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.122580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 247/493] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:48.123310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.165135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.166777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.167231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.167670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.181540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:48.191840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:48.192753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:48.193555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:48.196701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 247/494] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:48.197704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.237192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa801-69b0-4d20-9b4d-cede885b8f30 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.258599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.263887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.282905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.283584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.283860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.284160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.284915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:48.285217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.289110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.300853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 248/495] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:48.301680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.318216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843da61-eb7a-4a98-a3f9-0884e11905f9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.354085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.355110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.355393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.355660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.363335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.364753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-e717fa11-ec9f-4989-a755-75c57dc5dcd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:48.380407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:48.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:47:48.382243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:48.386440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 248/496] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:48.387628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8376f5-0c71-491f-b96a-f7929c961c8a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.387900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8376f5-0c71-491f-b96a-f7929c961c8a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.388201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8376f5-0c71-491f-b96a-f7929c961c8a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.389462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 249/497] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:48.393139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8376f5-0c71-491f-b96a-f7929c961c8a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.412042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ab9f-d91a-4a5d-b1fa-4c6e34ed1265 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.447919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.448595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.448984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.449259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.450114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:48.450429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.451795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.465674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 249/498] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:48.465674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.504151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.508941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8376f5-0c71-491f-b96a-f7929c961c8a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.529404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.544211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.545009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.545287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.545551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.553145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.554716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.566550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceba2f-b03d-4b9f-b8e7-cf93fffaae1c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2e1900c3-c2bc-40aa-b26b-bc6466cdaad2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:48.568482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:48.569371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:48.570301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 250/499] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:48.571066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabcaad-9fb9-4b79-8cd9-37270f1e5513 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.571354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabcaad-9fb9-4b79-8cd9-37270f1e5513 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.571626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabcaad-9fb9-4b79-8cd9-37270f1e5513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.588291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fcabcaad-9fb9-4b79-8cd9-37270f1e5513 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:48.589474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fcabcaad-9fb9-4b79-8cd9-37270f1e5513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:48.604025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.604697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.604973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.605225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.606060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:48.606364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.607807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.642731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 250/500] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:48.643499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.653664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 251/501] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:48.654989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde9709-ea6f-426f-8fa2-eddda8179a30 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.655279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde9709-ea6f-426f-8fa2-eddda8179a30 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.655554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde9709-ea6f-426f-8fa2-eddda8179a30 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.656844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 252/502] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:48.657537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde9709-ea6f-426f-8fa2-eddda8179a30 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.746178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.752578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecb9ad-7fb3-4e75-9ace-0696ab037030 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.762822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde9709-ea6f-426f-8fa2-eddda8179a30 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.773355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:48.785059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.785861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.786128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.786404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.796442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.797111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.797387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.797506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.797658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.798483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:48.798784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:48.799118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.800221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14f10d-eb4a-48d1-a1fb-890958737c6a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:48.816120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:48.817029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:48.817868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:48.817868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:48.817868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:48.817868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:48.817868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 251/503] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:48.818136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.842125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 253/504] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:48.843406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.843688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.843959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.862390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:48.863901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:48.920787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff1b14-f9f5-47be-b866-fee7b35ac548 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:48.929001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 254/505] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:48.929959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:48.954373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:48.955025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:48.955307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:48.955582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:48.973700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.024848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.025072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.026567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.028901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd83162-8e44-4f2d-9032-68c77ed48bf2 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.062673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.063465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.063749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.064054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.074018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:49.081619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:49.082690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 255/506] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:49.089167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.122806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 252/507] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:49.123756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.172727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.184093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47fc28-38d0-425a-9d11-8e0fba11f241 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.217385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.218044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.218324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.218595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.219325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:49.219632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.221065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.224728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a31da-11b0-4548-9148-51238876cf1d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.232491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 256/508] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:49.233281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.251065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.259412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.260085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.260367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.260639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.329426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.330772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19924b-326f-4fcb-9978-42c68a35ee0f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.330883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.363646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.364458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.364731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.364995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.375209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.376447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.377426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:49.393980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:49.394988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:49.395845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:49.395845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:49.395845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.395845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:49.395845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 257/509] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:49.396158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.434357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 253/510] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:49.435126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.464290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.495216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405714d0-ea5f-4957-9a03-4533c5768f2a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.529718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.530383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.530676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.530948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.531726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:49.532061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.533490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.537564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d471b-75e2-4d3c-a519-cfc5d34afbf9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.541831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 258/511] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:49.542612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.571679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.572575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.572785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.573057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.582764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.584267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.592807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:49.603180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:49.604186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:49.605286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:49.605286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:49.605286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.605286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:49.605286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 254/512] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:49.605286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.649401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24046c0-3bc9-4ee5-ad8b-d0d101408a50 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.677686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.683549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.684438 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.684662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.684939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.692511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.693985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.702720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f300f85-8583-4192-8a0d-9e5ed0a8decd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:49.708102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:49.709092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:49.710020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 259/513] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:49.710574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.741062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.741729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.742011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.742306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.743206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:49.743506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.744929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.755890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 255/514] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:49.756690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.811345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fe13e-06d5-4d88-b6f6-bc9ade728cee None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.816337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.845413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.846200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.846479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.846748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.857686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.858161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305fa2-abe8-4141-ae60-1c279301cb3d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:49.859135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:49.875706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:49.876819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:49.877677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:49.877677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:49.877677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.877677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:49.877677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 260/515] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:49.877677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.893999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:49.894801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:49.895085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:49.895359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:49.906110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:49.907594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:49.924167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:49.924651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:49.925620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:49.926667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:49.926667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:49.926667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:49.926667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:49.926667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 256/516] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:49.927317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:49.980863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cb35c-9a3c-449f-847d-e481ee2e799e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.014273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.014936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.015270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.015545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.016296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:50.016602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.017979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.024449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b105a-b2f8-4edd-855f-a97fdbf817ea None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.030870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 261/517] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:50.031718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.038668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.062050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.062715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.062995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.063266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.064129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:50.064433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.065889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.082831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 257/518] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 09:47:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:50.083653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.133484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1d0a9-02ad-4bd9-a782-d8cc7782deb0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.136509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.168059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.168730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.169005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.169277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.169980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.171434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.189080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e5239-78b5-4029-9421-cf1095fb6025 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.193048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 262/519] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:47:50 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:47:50.193886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.236176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.237509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.237509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.237509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.238132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:50.238436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.239847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.254386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 258/520] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:50.255118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.294690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.317604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b81f0-50ec-44ce-a60b-786829d1c326 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.355181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.355864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.356168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.356441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.357591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:50.357591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.358874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.371837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058a86-81a9-49cf-a811-ac3479e94b8c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.372948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 263/521] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:50.373392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.397544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.421675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.423008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.423383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.423748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.431377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.433047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:50.447998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:50.449056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 259/522] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:50.450094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.496230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a84f-794c-4e96-8503-6bdc6b7b4414 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.532691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.533468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.533786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.534115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.544720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.556916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84a3fb-e95b-46e6-970f-f3bc55ccd1da None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.599704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.600914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.601837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.602250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.603406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.604910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.610860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.612488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.625562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:50.626930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:50.628208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:50.629585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:50.633215 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 260/523] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:50.634191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.724137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 264/524] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:50.724137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.752235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f7b33-9d4a-4075-b17a-4ee71f7c4e26 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.787309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.788291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.788669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.789053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.796996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.798560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:50.814537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:50.815656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 09:47:50.816798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:50.820483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 261/525] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:50.821952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.830159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d38cb-8320-44df-87bf-998a5697f7b2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.837320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.853555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:50.869060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.869727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.870005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.870274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.871063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:50.871372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.872856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:50.895423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 265/526] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:50.895423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:50.943087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da334a-1f7d-4921-b5bc-eff7c18f665c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:50.977973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:50.979083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:50.979371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:50.980067 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:50.981140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:50.981462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:50.983616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.001354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d5292-2fb4-492e-98eb-e4d8a81399bb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.004961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 262/527] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:51.005796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.042085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.042935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.043215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.043481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.053951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.055468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:51.072848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:51.073882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:51.079831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:51.079831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:51.079831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:51.079831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:51.079831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 266/528] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:51.079831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.095565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.105922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.130984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6cc66-e105-4ecf-ae90-aec4e6763112 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.170067 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.170734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.171018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.171292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.172158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:51.172463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.173913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.178190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebb8e3-0fde-457f-b6f7-a196d38a4046 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.191488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 263/529] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:51.192411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.218770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.219437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.219812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.220019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.220863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=14dd1de1fe914f319017818e571732de) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:51.221264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.222635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.235901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 267/530] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/role_assignments?user.id=14dd1de1fe914f319017818e571732de&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:51.235901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.333355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08fdc2-1d5d-4af3-8cba-0c04d05836d1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.360051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.370229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44035-df81-41e4-ad23-9b53002f7a20 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.377625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.389126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.389864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.390145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.390468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.391337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=0a98b63a4ad14d7399938ffb9c6b018a) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:51.391650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.393187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.400433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 264/531] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/role_assignments?user.id=0a98b63a4ad14d7399938ffb9c6b018a&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:51.400622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.423609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.424656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.425115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.425586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.427095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:51.427893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.430350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.447930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 268/532] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:51.453804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.533998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa634f4-7593-40ea-b953-084bd17d9ff5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.571507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.572103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.572442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.572717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.581328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a84ab-9eeb-4d23-9cd5-7aaa7584e6b8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.635639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.637076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.637369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.637879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.648032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.656749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.658236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.667670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.669252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:51.680127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:51.681079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:51.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:51.690572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 269/533] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:51.696141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.719484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.796627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 265/534] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:51.809003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.862102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8da89-8167-452f-aca0-8e02122c2a21 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:51.900091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:51.900938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:51.901225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:51.901504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:51.902310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:51.902623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:51.905049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:51.927937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 270/535] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:51.936306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:51.963337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:51.968913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27702-67c1-4879-afe7-c38765601b44 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:52.040116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:52.041350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.041847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.042347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.061945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:52.064341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:52.096596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:52.108863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:52.110112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:52.110944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:52.110944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:52.110944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:52.110944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:52.116584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 266/536] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:52.121632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:52.146424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3131d31-376e-437b-ba40-a88f9dd4c363 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:52.203116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:52.204192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.204684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.205151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.206197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=15924f58d53145ae83b3b4cfbbb02ef7) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:52.206716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:52.208396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:52.215200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 271/537] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:51 2022] GET /identity/v3/role_assignments?user.id=15924f58d53145ae83b3b4cfbbb02ef7&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:52.317544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19832ba-d2e8-4716-9775-069a8bd572e0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:52.353059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:52.353730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.354015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.354293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.355031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:52.355336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:52.356755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40e1c9-2fef-4683-b918-7759656de384 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:52.369826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 267/538] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:52.374956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:52.441179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026b11-ae2d-4b58-a459-b3f922ab08cd None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.441496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026b11-ae2d-4b58-a459-b3f922ab08cd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.441739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026b11-ae2d-4b58-a459-b3f922ab08cd None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.444026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 268/539] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:52.449014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.449295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.449570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.462190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:52.474894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:52.476214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:52.525763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5288f2-9ed8-4f54-8a16-da5852b78e41 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:52.552383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 269/540] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:52.562536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:52.563345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.563627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.563917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.571059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:52.572585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:52.572938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:52.575250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:52.588983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:52.589878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:52.590700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 272/541] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:52.596044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:52.678355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692ecf6-f9ab-413d-ae0e-57a4d82ce0d9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:52.716845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:52.717736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.718015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.718287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.720468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:52.742827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:52.750822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:52.751928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:52.752826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:52.768256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 270/542] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:52.771387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:52.786387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6fbc7-5c24-4ce6-ad64-8b779bc21adc None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:52.854358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:52.855274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:52.855697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:52.856133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:52.857083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:52.857576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:52.859168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:52.889645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 273/543] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:52.893960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:52.977533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fed73-d092-4d74-9559-c51df8bcde06 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:53.021936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:53.022502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:53.023015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.023546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.024091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.025050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:53.025529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:53.027090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:53.041429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 271/544] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:53.043140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:53.081722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74028570-923a-47d1-8fb2-3769fa431107 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:53.102346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:53.144598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:53.145276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.145563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.145845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.146694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=7b086db83a6b47fb98d450bb7e1c8026) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:53.147013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:53.148711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:53.151041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9d0d-ea1c-4b6c-80d2-b19b98514d97 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:53.164916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 274/545] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:52 2022] GET /identity/v3/role_assignments?user.id=7b086db83a6b47fb98d450bb7e1c8026&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:53.188949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:53.190105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.190555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.191036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.204011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:53.205826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:53.226160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:53.227193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:53.228194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:53.228194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:53.228194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:53.228194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:53.234888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 272/546] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:53.241058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:53.401314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3958-5f16-4739-b99b-61bde1b195ce None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:53.429836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:53.449575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:53.450357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.450641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.450916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.451661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:53.451996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:53.453395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:53.467009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 275/547] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:53.472550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:53.616552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1156f6a-ab69-4c40-acfd-58286e9902d0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:53.626210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:53.661025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:53.661860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.662149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.662428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.675498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:53.677054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:53.696143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:53.697144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:53.698532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 273/548] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:53.703582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:53.824377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7825063-d3d8-4d42-9733-cd945c7b0c8f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:53.865497 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:53.866181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.866464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.866798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.868299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:53.868618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:53.870073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b87b-b44d-47aa-a758-da132cf3ad92 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:53.894362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 276/549] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 09:47:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:53.900023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:53.925912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21df8b9-5890-4b93-8690-c18e812800b4 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.926223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21df8b9-5890-4b93-8690-c18e812800b4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.926479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21df8b9-5890-4b93-8690-c18e812800b4 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.928159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 277/550] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:53.933957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f0c0-7370-4edd-b55b-377857b1e891 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:53.934245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f0c0-7370-4edd-b55b-377857b1e891 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:53.934527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f0c0-7370-4edd-b55b-377857b1e891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:53.955026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2292f0c0-7370-4edd-b55b-377857b1e891 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:53.956424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2292f0c0-7370-4edd-b55b-377857b1e891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:54.020302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.031899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 278/551] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:47:54.064879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac974d-7244-47a9-b2be-98363165fd33 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.065195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac974d-7244-47a9-b2be-98363165fd33 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.065433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac974d-7244-47a9-b2be-98363165fd33 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.067150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 279/552] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:54.067865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac974d-7244-47a9-b2be-98363165fd33 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.093302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6406a-3154-4f29-9384-cc967b8f7bf6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.130920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.132075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.132644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.133174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.171505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.193042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac974d-7244-47a9-b2be-98363165fd33 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.213683 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:54.215499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:54.237474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad787-2a5c-4fc6-8e70-402c5733b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.238338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad787-2a5c-4fc6-8e70-402c5733b567 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.238663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad787-2a5c-4fc6-8e70-402c5733b567 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.238954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad787-2a5c-4fc6-8e70-402c5733b567 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.253180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dad787-2a5c-4fc6-8e70-402c5733b567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-64dad787-2a5c-4fc6-8e70-402c5733b567 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:54.262811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:54.263861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 280/553] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:54.271479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.273151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.273151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.298577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:54.299788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:54.329127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.362474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 274/554] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:47:54.363684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.388078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 281/555] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:54.388934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.453820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.490559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bac1d-615a-4fec-a649-23d8fd580361 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.517189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b200b58-6256-4a82-a22d-e0db69f7e736 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.525482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.526167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.526446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.526848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.527610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:54.527934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:54.529333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:54.542229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 275/556] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:54.543146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.560498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.561668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.562291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.562896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.582452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:54.584429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:54.606703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:54.607720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:54.608639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:54.608639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:54.608639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:54.608639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:54.611026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 282/557] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:54.612059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.631763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.652629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6044da-4370-4ccc-bb1c-1025af5b15f5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.681978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.688519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146802fb-e7ab-4678-bcce-c774de1a6884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.689443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146802fb-e7ab-4678-bcce-c774de1a6884 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.689621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146802fb-e7ab-4678-bcce-c774de1a6884 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.689865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146802fb-e7ab-4678-bcce-c774de1a6884 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.700277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146802fb-e7ab-4678-bcce-c774de1a6884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-146802fb-e7ab-4678-bcce-c774de1a6884 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:54.707807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:54.709288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:54.710109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 276/558] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:54.710857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf560e0-bf9d-47c1-a109-90d1215f93b2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.710857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf560e0-bf9d-47c1-a109-90d1215f93b2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.710857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf560e0-bf9d-47c1-a109-90d1215f93b2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.716466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 277/559] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:54.717220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf560e0-bf9d-47c1-a109-90d1215f93b2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.726229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd388165-910b-499d-8c0a-9c5f27c3fcc3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.761496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.762293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.762576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.762845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.773476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:54.774971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:54.792278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:54.793173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:54.794131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:54.794131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:54.794131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:54.794131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:54.798510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 283/560] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:54.799254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.824743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf560e0-bf9d-47c1-a109-90d1215f93b2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.861359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.862200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.862574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.862913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.863712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:54.864108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:54.865549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63b3a5-a5e1-49b2-9b0c-13752a8618ab None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:54.872712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.877936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 278/561] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:54.882526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.882526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.882526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.897949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:54.899382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:54.930295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81894a6c-8014-423a-9601-ef3a6f031d64 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:54.958825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:54.971618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:54.972339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:54.972624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:54.972916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:54.973724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:54.974016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:54.975481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:54.984392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 284/562] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:54.985224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 279/563] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:54.986200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:54.992272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.099975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98fde1-fac4-4bfa-897c-a4e1f9f37083 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.106399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e75897-7c63-4af3-8d4d-7272b6cb0f3c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.136016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.138869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.139682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.139997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.140278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.145325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.145994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.146275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.146555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.147276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:55.147448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.147602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.149028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.149131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.165712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 285/564] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:55.165712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:55.165966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:55.167597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 280/565] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:55.168556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.213608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.276604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e80a1-6b57-4bd6-b0c9-11717e9bd681 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.280345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd63615-0223-4f0a-b16f-441ca7886330 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.312618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.313426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.313709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.314001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.319047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.319848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.320182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.320456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.325052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:55.331244 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.332729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:55.332953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:55.333872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:55.334705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 281/566] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:55.335417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:55.355347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:55.356558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:55.357495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:55.357495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:55.357495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:55.357495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:55.357495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 286/567] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:55.357495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.406261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.449389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.452117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7683984-dd0f-49ed-9ca1-1567e6e2428c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.468623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904faebd-9a3b-4134-ab0e-1b77198e8ff6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.487043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.487847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.488143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.488411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.496128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.497575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.502107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.502772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.503052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.503329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.504153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:55.504467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.505920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:55.511319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:55.512497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 282/568] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:55.513418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.519373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 287/569] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:55.520237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.611238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ddea-2ab7-424e-9297-51dd10f5cbcf None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.618827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.621938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ac2f-5a60-4c4f-ad74-591d83e97b2d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.645628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.646286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.646572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.646852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.647606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:55.647941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.649333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a203-0418-464e-9351-4abc95e18cc9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.655292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.655980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.656262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.656532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.657255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:55.657341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 283/570] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:55.657560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.658550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ec0f8-a51f-4cf8-b560-e362e6350017 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.658834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ec0f8-a51f-4cf8-b560-e362e6350017 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.658923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.659110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ec0f8-a51f-4cf8-b560-e362e6350017 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.660532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 284/571] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:55.661033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ec0f8-a51f-4cf8-b560-e362e6350017 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.676314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 288/572] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:55.676983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.689925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.767103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ec0f8-a51f-4cf8-b560-e362e6350017 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.772408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3e02-bdd4-4a92-84f2-f96f3abb4d5a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.799865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.800540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.800817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.801078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.801915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:55.802217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.803648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d2262-1c44-49e7-af1c-b71633ab49b6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.805653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.806304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.806581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.806852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.807679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:55.808029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:55.809448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:55.815025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 289/573] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/role_assignments?user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:55.815967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.821405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 285/574] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:55.822759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.823037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.823311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:55.842367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:55.843603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:55.857282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.923596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 286/575] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:55.923596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:55.942317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:55.957949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02e66e-2725-4a7b-8f07-30b3a54e7b58 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:55.997299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:55.998349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:55.998784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:55.999054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.010037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.011486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:56.029460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.030357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.032247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:56.032247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:56.032247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.032247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:56.032247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 290/576] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.037071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.045007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c277999-d6d5-4668-ad8d-06f7c453061d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.081303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.082260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.082378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.082638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.091054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.092571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:56.107479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.108490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:56.109448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 287/577] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:47:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.110677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.123557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.147836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf94665-7fc4-41df-b963-d2ffc8d66e52 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.182556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.183551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.185421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.185700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.185941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.196517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.197988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.216120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b0fe0-cc1c-4b6d-98a9-1f2a40ba2e55 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:56.219100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.220259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.226116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:56.226116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:56.226116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.226116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:56.226116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 291/578] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.226116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.252013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.252823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.253114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.253393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.265717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:56.273501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.274449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:56.275278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 288/579] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.275974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.339687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f5d68-5f97-48b6-8f06-69b8297630c0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.367122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.376112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.376783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.377059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.377334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.378062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:56.378363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.379755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.383321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a283798-c9e0-488a-9d09-9391f1496163 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.391919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 292/580] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:56.392082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.418444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.426088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.426762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.427034 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.427310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.428183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:56.428490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.430308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.453525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 289/581] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:56.453525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.500313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e95b9-8335-4a41-848b-54a2d614384f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.535994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.537404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.538008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.538257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.539302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:56.539608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.541183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.554091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 293/582] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:56.554984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.566995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30653-ff05-4edb-96cd-063f5697c6c8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.604024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.605275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.605950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.606257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.606534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.607441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:56.607747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.609174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.618328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 290/583] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:56.618947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.661246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d562-7e7d-45a6-824c-e7e5b9682da6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.668382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.697133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.697928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.698211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.698506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.705759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.707422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.719486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb5623-04e1-4f25-bcf0-c0e86b8cd3c7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:56.722401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.723297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:56.724251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 294/584] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.724821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.755118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.756247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.756526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.756800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.768259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.769719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:56.787124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.788998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:56.788998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:56.788998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.788998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:56.788998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 291/585] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.789681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.836006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f241-0a06-401e-86cf-cc1bf5945509 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.846979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.873507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.874311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.874592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.874860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.882102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.883650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:56.897429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.898436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:56.899319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:56.899786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b4996-4b08-418b-b421-c7d31fc0c015 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:56.900914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:56.903694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 295/586] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.904728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:56.938769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:56.939573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:56.939854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:56.940160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:56.951322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:56.952840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:56.970547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:56.971589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:56.972535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:56.972535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:56.972535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:56.972535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:56.972535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 292/587] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:56.972971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.019802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b640ce-fc75-4df8-a4b5-85033d95f885 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.056997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.058016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.058467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.058908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.059977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:57.060353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.061803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.078114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e027c3-72f7-4db6-8a93-60ed5b9044ab None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.085175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 296/588] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:57.086141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.114092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.115460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.116325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.116597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.116988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.117803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:57.118085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.119438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.131811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 293/589] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:57.132833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.155146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.198251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b7d25-ee6e-407d-9030-6879142775ef None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.233629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.234289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.234579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.234852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.235673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:57.236007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.237489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.241187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cd8c7-bcbb-4ceb-b8aa-6b74afc1f7fd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.254411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 297/590] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:57.254411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.282506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.283179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.283465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.283736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.284493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:57.284800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.286179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.300976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 294/591] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:57.304987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.347612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.364144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2997-5808-42b3-960c-2ae0d0a0b251 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.388949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.399623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.400850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.401133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.401407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.411949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.413424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.417095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b091-0d3f-4822-b48c-c0a2a53f3c19 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:57.433859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:57.434919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:57.436026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:57.436026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:57.436026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:57.436026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:57.440013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 298/592] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:57.441035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.457350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.458299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.458581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.458858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.466347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.467823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:57.483010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:57.483962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 295/593] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:57.485359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.557644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b3f79-a455-4f83-82b6-e33863ca8cae None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.592645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5ef8b-0674-4a0a-ab47-0dce02292f39 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.595504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.595844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.596156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.596432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/0a98b63a4ad14d7399938ffb9c6b018a/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.606784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.614523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=0a98b63a4ad14d7399938ffb9c6b018a, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.615907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.630871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.631524 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 299/594] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:57 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/0a98b63a4ad14d7399938ffb9c6b018a/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:57.632207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.636453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.636651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.636863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.646269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.657211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.660037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:57.682114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:57.683151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 296/595] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:57.690120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.782237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b34f7e-82e1-46f9-8ac5-f1c5481d1ac3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.802665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84561fa5-e42a-4eaa-8604-62705c99fd17 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.829029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.830299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.831191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.831938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.833176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:57.834139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.836092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.842716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:57.843386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.843674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.843984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.844785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:57.845098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:57.846537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63b994-8348-4481-95ac-5038390ca388 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:57.849969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 300/596] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:57.850811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.860449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 297/597] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:57.861933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846f34f-c31b-4e1b-93eb-a215e8717999 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:57.862770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846f34f-c31b-4e1b-93eb-a215e8717999 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:57.863190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846f34f-c31b-4e1b-93eb-a215e8717999 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:57.865199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 298/598] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:47:57.869795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846f34f-c31b-4e1b-93eb-a215e8717999 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:57.905139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.927652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:57.963636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6180-b467-4f31-a59f-7adbb9ab8728 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:57.974929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846f34f-c31b-4e1b-93eb-a215e8717999 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.001770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.002787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.003068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.003345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.004329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:58.004618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.006173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.009943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.010779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.011060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.011323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.018389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.019825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.031338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 301/599] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:58.031338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-a9297730-4712-499a-9f44-0f297c56920e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:58.033921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:58.034912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:58.035778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 299/600] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:58.036733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.036733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.036957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.053773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:58.055061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:58.128008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 300/601] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:47:58.128797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.140327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d630f-7838-4323-9e13-b0cf8266a91a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.149610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:58.179673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.180776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.181047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.181316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/14dd1de1fe914f319017818e571732de/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.189265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:58.197005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=14dd1de1fe914f319017818e571732de, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.198478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.219909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 302/602] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:58 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/14dd1de1fe914f319017818e571732de/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:58.219909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.268240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404013b-80e8-43f3-b491-c5d7bb872636 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.320385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.321666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.322879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.323364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.339158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.341758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.349286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aacb94-2f7f-4523-a6ba-4731c69a4c34 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:58.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:58.368548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:58.369414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:58.369414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:58.369414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:58.369414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:58.372315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 301/603] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:58.373484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.403484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.404123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.404426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.404674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.405479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:58.405783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.407241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.422416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 303/604] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:58.423820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.448448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:58.492618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:58.498523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161e6a6-d322-4547-8901-0daba4b9d7f5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.532810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.533879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.534301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.534663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.546877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:47:58.551259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0875325-a416-44d3-9844-3a9e13ffbc18 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:47:58.556780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:58.557881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:47:58.558903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:58.560441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 302/605] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:58.561217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.596916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.598660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.599052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.599428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.600306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:58.600712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.602717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.618869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 304/606] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:58.619940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.670545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae943b0-e5f9-43f1-8dca-fa3518f2f2a1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.710086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.711032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.711640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.711951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/15924f58d53145ae83b3b4cfbbb02ef7/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.720681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:58.729661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=15924f58d53145ae83b3b4cfbbb02ef7, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.731550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.753145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db0ef3-7b37-444d-9c3f-36accbe5b3aa None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:58.754524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 303/607] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:58 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/15924f58d53145ae83b3b4cfbbb02ef7/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:58.755901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.772277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:58.823413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:58.827942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:58.827942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:58.827942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:58.827942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:58.827942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:58.827942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:58.849453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 305/608] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:58.861609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:58.973970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267a12-16f7-439f-ba4e-77837c38d2ea None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.010500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76c1b-382b-47f8-bdf4-fbc483489f02 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.015746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.016995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.017282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.017556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.025782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.027270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:47:59.043643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:59.044697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:47:59.045579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 304/609] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:59.052906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:59.096606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.097433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.097717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.097994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.118856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:59.122281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:59.125827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.127322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.160224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c998cb5-0a0c-464b-8710-98a596a89460 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:47:59.168133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:59.169019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:59.169911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:47:59.169911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:47:59.169911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:47:59.169911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:59.169911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 306/610] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:47:59.175245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:59.195625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.196497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.197035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.197320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.198191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:59.198507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.200075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.215312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 305/611] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:47:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:59.221072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:59.364166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683afb0-1b06-432a-a632-80f10068ed44 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.373531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26605217-9077-451a-a632-2e108e6a4cc0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.415323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.416041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.416327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.416604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.417623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:59.417935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.419163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:59.420162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.436141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 307/612] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:47:59.445148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.446395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.446677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.446955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/7b086db83a6b47fb98d450bb7e1c8026/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.448025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:59.481349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=7b086db83a6b47fb98d450bb7e1c8026, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.482827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.515780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 306/613] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:47:59 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/7b086db83a6b47fb98d450bb7e1c8026/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:47:59.540373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:47:59.590543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2678bc-1040-4c1c-b25a-846da4cdbd0e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.629629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.630446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.630734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.631015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.640200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.641742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:47:59.658797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:47:59.659809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:47:59.660768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 308/614] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:47:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:47:59.666181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:59.798047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12fe9c-a5e7-4e26-b8ec-51ebfbf6a686 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:47:59.834403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:47:59.835083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.835362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.835713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.836923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:47:59.837417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:47:59.838927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a56ce-fb0b-4eb8-a752-a2fa6d44ee52 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:47:59.856845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 307/615] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 09:47:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:47:59.862383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:47:59.867223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd07e8-9b50-42c0-887a-7feef148a310 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.867747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd07e8-9b50-42c0-887a-7feef148a310 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.868222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd07e8-9b50-42c0-887a-7feef148a310 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.869842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 308/616] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:47:59.875030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:47:59.875315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:47:59.875591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:47:59.907449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:47:59.911054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:47:59.964170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:00.034095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 309/617] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:48:00.045223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e6eec-5bf7-4184-b7d1-2d4f47aa92e6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.065330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:00.091998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.110171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.110896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.111171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.111441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.130514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:00.132044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:00.162273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:00.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:00.164259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:00.164259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:00.164259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:00.164259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:00.165556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 309/618] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:00.169564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.234312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cdf57-bc9b-47fc-85ec-e5b4db3e5f0b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.279810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.280651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.280935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.281208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.308481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:00.316138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:00.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:00.317866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:00.323945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 310/619] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:00.324861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.345273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eccd4-8881-494a-b17f-333917be1475 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.380841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.381428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.381704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.381974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.382775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:00.383073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:00.384494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:00.387981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:00.396976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 310/620] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:00.403627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.454862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:00.538960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63753d-d29b-4c32-abf6-6f9c01f59a79 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.574858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.575708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.576020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.576289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.583887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:00.585544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:00.600821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:00.602002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:00.602878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 311/621] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:00.608041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.609953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeded6dc-a144-4faa-8e49-2c770f58a2ce None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.659351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.659984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.660268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.660542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.661304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:00.661602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:00.662991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136a2a3-2336-4ceb-97a3-c3bc9519fd38 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:00.672027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 311/622] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:00.673282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6692f-17f0-46e1-9b4d-229d9c1b805a None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.673500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6692f-17f0-46e1-9b4d-229d9c1b805a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.673836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6692f-17f0-46e1-9b4d-229d9c1b805a None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.679304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 312/623] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:00.680521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6692f-17f0-46e1-9b4d-229d9c1b805a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.707042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:00.732349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed251c8-cbdd-4f1d-b112-e7b88db8cd9a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.777119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:00.783460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.784608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.785283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.785704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.787226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:00.787780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:00.790090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2a986-638e-40d2-b09f-3236320123a2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:00.796610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6692f-17f0-46e1-9b4d-229d9c1b805a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:00.806251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 312/624] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:00.807799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.808243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.808645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.831577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:00.833550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:00.839235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:00.840226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:00.840651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:00.841024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:00.852188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:00.853802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:00.871931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:00.873064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:00.874159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:00.874159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:00.874159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:00.874159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:00.875711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 313/625] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:00.876797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.917803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 313/626] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:00.918588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:00.992298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.010952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b049b-5e04-4325-8dd1-2c9ea1fe2904 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.021961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3128a-da4f-4cbd-9b43-6c81377497c8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.048583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.049656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.050090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.050476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/64bf7840b5fd4b3b9dd2b8e18b4495b3/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.057644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.058312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.058590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.058863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.059586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:01.059907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.060425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.061317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.068292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=64bf7840b5fd4b3b9dd2b8e18b4495b3, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.069890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.077925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 314/627] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:01.078869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.096099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 314/628] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:00 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/64bf7840b5fd4b3b9dd2b8e18b4495b3/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:48:01.097312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.184772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec17b-2107-477b-b169-ecfcc43732d2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.216448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.225054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689ab23-7896-4696-b6fc-536d099e747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.225881 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689ab23-7896-4696-b6fc-536d099e747c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.226190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689ab23-7896-4696-b6fc-536d099e747c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.226462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689ab23-7896-4696-b6fc-536d099e747c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.233143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c79ee-6051-445a-aad6-11de8c6f7034 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.242157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689ab23-7896-4696-b6fc-536d099e747c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-f689ab23-7896-4696-b6fc-536d099e747c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:01.254301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:01.255450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:01.257286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:01.260473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 315/629] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:01.262842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409594a5-4e3d-4e7d-b609-4e97a8898dd4 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.263525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409594a5-4e3d-4e7d-b609-4e97a8898dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.264360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409594a5-4e3d-4e7d-b609-4e97a8898dd4 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.270605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 316/630] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:01.270605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409594a5-4e3d-4e7d-b609-4e97a8898dd4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.277353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.278552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.279036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.279556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.288794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.290732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7988f39e-df96-46c8-b7ea-10190e686828 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:01.308241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:01.309309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:01.310685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 315/631] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:01.311492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.311969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.314099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.330383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:01.331557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:01.382788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.413667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409594a5-4e3d-4e7d-b609-4e97a8898dd4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.419659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 316/632] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:01.421237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.470961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.471549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.471830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.472129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.473769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:01.474082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.475463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.490899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 317/633] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:01.490899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.551909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.565332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638c29c-93f8-4753-ba06-e39331a53872 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.608704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.610338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.610795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.611237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.612173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7115a-8abe-414c-8f6b-3c83959b5db6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.612483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:01.612968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.615694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.637254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 317/634] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:01.638385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.648358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.649130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.649478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.649821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.661014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:01.669085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:01.670051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:01.670899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:01.675391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 318/635] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:01.676331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.681162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.754298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c966-f47f-43be-be32-b2736ab090ec None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.780012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7966c-dae9-454b-abb7-1ed736aa1670 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.792794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.794026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.794491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.795690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.796198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.809368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.811294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.815323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.816082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.816365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.816639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:01.832298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:01.833270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:01.834149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:01.834149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:01.834149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:01.834149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:01.837227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 318/636] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:01.838286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:01.900133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.901650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:01.947695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:01.960968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a1dd1-6a4b-499c-844b-d368d4ae03bc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:01.995986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:01.996659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:01.996937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:01.997207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:01.998065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:01.998484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:01.999855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196f3e4-861a-4221-8785-db45cb8858f0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.001830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 319/637] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:02.002669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.013247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 319/638] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:02.014821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2671960-5b23-44bc-a7d5-2eceb05f35ac None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.015133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2671960-5b23-44bc-a7d5-2eceb05f35ac None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.015408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2671960-5b23-44bc-a7d5-2eceb05f35ac None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.016789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 320/639] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:02.021373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2671960-5b23-44bc-a7d5-2eceb05f35ac None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.023544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.113144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c685f9-a3e5-4a0a-b25e-68f297159fdf None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.138010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2671960-5b23-44bc-a7d5-2eceb05f35ac None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.150542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.151317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.151663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.152027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.152822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:02.153187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.154626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.171101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 320/640] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:02.171101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.179454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.180339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.180623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.180898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.191572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.193168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-45c45fd9-d8fa-4a6d-906c-394220eba505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:02.212200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:02.213137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:02.214024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:02.214024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:02.214024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:02.214024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:02.214024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 321/641] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:02.215090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.215090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.215261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.220155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.233825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:02.234996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:02.251613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.277801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b7d81-2e7b-4111-903f-ceca2e077cdb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.314065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.324052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 322/642] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:02.324204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.324204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.324204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.324459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.337131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.338690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:02.357657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:02.358998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:02.360283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:02.360283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:02.360283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:02.360283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:02.360283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 321/643] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:02.360283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.437316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429e291-86f4-412d-9cd9-5d9594c3a842 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.468024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.474647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.475448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.475729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.476030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82abe7d-93db-4bf2-9cd5-10779931e194 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.476166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.484711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.486529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.501634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:02.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:02.504771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 323/644] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:02.511327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.513461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.514530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.515152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.515813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.516567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:02.516868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.518280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.530743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 322/645] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:02.532304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.614230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddba9c-8419-4935-9d93-21112a65c692 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.635978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23a4a5-9c85-47f0-8416-0374ebeb2476 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.654553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.655572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.656184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.656464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.667597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:02.669675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.670452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.670738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.671010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.671730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:02.672063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.673437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:02.675443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:02.676409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:02.677248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 324/646] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:02.678593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.691386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 323/647] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:02.692130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.736953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.747094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.790027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821532b1-1305-472d-a54d-ccb3d43f7585 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.801915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e58cb-72c2-4909-b370-f67b08ec9b54 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.825396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.826070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.826356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.826634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.827590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:02.828040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.829480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.835120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:02.836064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:02.836678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:02.836945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:02.843686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:02.845183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:02.853076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 325/648] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:02.853076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:02.861987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:02.862894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 324/649] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:02.864544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:02.960463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052013a-13c0-46d5-96c2-1c4f2e7acd35 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.965947 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be999d7a-c5f2-4ca5-a71e-8ea2ca313a6c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:02.972927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:02.973460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.000719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.001447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.001651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.002030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.002603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.002790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:03.003089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.003442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.003719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.004015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.004512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.012471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.013799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 326/650] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:03.014091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.018907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:03.030455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:03.031643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 325/651] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:03.033375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.126977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e478b7-c9a2-46f3-966a-f395feafc7b8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.140123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f10d-7e64-406e-8b0f-fd77bf751b71 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.162819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.163620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.163919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.164204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.178418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.179695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.180236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.180805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.181770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:03.182343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.184067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.196519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 326/652] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:03.197334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.205247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.223714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.232473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.233940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.295320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56872ce7-52d1-4a5c-bb99-24ac9cac2236 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.328398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.329195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.329473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.329752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.333105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 327/653] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:03.333945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.342262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.343759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-e968138a-a71f-42c7-96dc-de57362d8b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:03.361084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:03.362097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:03.362984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:03.362984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:03.362984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:03.362984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:03.362984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 327/654] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:03.363557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332acd-8348-461e-86b4-c59b0900e76f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.363784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332acd-8348-461e-86b4-c59b0900e76f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.364084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332acd-8348-461e-86b4-c59b0900e76f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.365676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 328/655] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:03.366373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332acd-8348-461e-86b4-c59b0900e76f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.410943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.439772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b3393-79d9-4294-a081-0843ded02119 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.445221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.469115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332acd-8348-461e-86b4-c59b0900e76f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.473534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.474208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.474489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.474762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.475604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:03.475994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.477436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.494541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 328/656] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:03.495364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.507074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.507778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.508106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.508381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.509207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=0a98b63a4ad14d7399938ffb9c6b018a) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:03.509522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.510957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.517060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 329/657] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/role_assignments?user.id=0a98b63a4ad14d7399938ffb9c6b018a&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:03.517831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.606218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97783522-2782-400c-91c9-4050f0dd6691 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.638583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5de51-40c2-42da-a45a-22a639ba900b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.641758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.649126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.649943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.650223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.650498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.661969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.663402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.676923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.677862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.678143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.678415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.679248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:03.680220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.682048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-48911f6e-b199-4d56-b69d-4a6fafd8df70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:03.683389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:03.684262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:03.685153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:03.685153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:03.685153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:03.685153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:03.685153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 329/658] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:03.686295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.686295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.686549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.696459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 330/659] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:03.696635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.699304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.705413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:03.706548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:03.779491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 330/660] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:03.783940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:03.808418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99a6c-328f-4b5e-9c5d-ab75850eda47 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.844188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.844846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.845125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.845398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.888453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fcaaa-4433-4728-a5a6-68468d6bf806 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:03.903122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.918415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.920233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.922486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:03.929471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:03.930233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:03.930516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:03.931580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:03.932291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:03.932602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:03.934326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:03.947251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 331/661] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:03.951726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.032767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 331/662] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:04.037955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.061480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcf406-3112-48b5-8696-362775aba055 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.099320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.100316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.100440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.100700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.109427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.110892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:04.125511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:04.126460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:04.127391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 332/663] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:04.128064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.152745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac3c25-b938-49b5-b010-bea868b3cd12 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.154397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.168991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.190911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.191763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.192111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.192384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.203649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:04.211740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:04.212668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 332/664] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:04.218667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.244881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9161091-c6db-4f53-bafb-22797cbd0cab None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.290989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.292137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.292299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.293113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.302019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.303903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:04.321318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:04.322258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:04.323122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 333/665] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:04.323602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.331077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512547a-a5b7-48aa-95cc-8cf8867cb54a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.368460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.369266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.369548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.369823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.380682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.382672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:04.399713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:04.400760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:04.401650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:04.401650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:04.401650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:04.401650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:04.401650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 333/666] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:04.402758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.417928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.422802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.431655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61ccf7-0cb5-4bf5-a78a-a460e134a849 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.466645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.467392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.467638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.467943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.468787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:04.469103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.470563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.488474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 334/667] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:04.489278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.515134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493ae6-0461-487b-a3a7-40c067292fe2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.549659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.550328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.550606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.550875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.551624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:04.551952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.553321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.561780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 334/668] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:04.562381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.599568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a0c4c-d43f-4a25-9cc4-528101af2730 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.634166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.634983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.635332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.635696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.636570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:04.636946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.638415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.648961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.653578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 335/669] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:04.653578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.668285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.679854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9a79-78a6-4e7c-b283-a6cc434a156d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.717178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.717856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.718142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.718421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.719158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:04.719469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.720894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.737925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 335/670] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:04.737925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.771720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21e3ef-7d87-4d5e-abbc-d87042e746d3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.807348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.808036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.808317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.808593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.835846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c9032-82d8-4ad1-aa8f-e470dde150db None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:04.874034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:04.874826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:04.875095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:04.875378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:04.878095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.878465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.880388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.890349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:04.891801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:04.891952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:04.909846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:04.910929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:04.911839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:04.911839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:04.911839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:04.911839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:04.911839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 336/671] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:04.912163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:04.989904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 336/672] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:04.990935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.020703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebf652-ff42-4813-af40-958dedb376d8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.057428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.058107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.058374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.058651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.059481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=14dd1de1fe914f319017818e571732de) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:05.059789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.061266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.075903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 337/673] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/role_assignments?user.id=14dd1de1fe914f319017818e571732de&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:05.075903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.111776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42de940-4177-4d5d-8d6b-d494e489c885 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.124043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:05.133927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:05.165645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.167459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.168082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.168555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.178608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.181081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.194647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f354c-7263-4e3b-9dff-84b149df93af None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:05.196221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:05.197098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:05.198001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:05.202597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 337/674] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:05.203719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.240045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.241138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.242000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.242494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.243446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:05.243983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.245760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.259337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 338/675] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:05.260692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.347269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7f3a-2556-469a-baef-2132c32f52a2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.388661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.389271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.389556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.390465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.393063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bef400-0deb-4f1f-b306-df545d6087bd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.402654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.404209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.406845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:05.422812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:05.423766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:05.424690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:05.424690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:05.424690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:05.424690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:05.424690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 338/676] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:05.425412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.436490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.437221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.437509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.437788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.438596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:05.438910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.441012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.457867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:05.459531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 339/677] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:05.460561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.540144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0c0e-b450-4b38-ae30-d0c43c95503a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.585154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335809b-5a11-49e3-8880-a9a6f8528475 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.585416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.586507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.586803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.587081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.597779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.599503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:05.615655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:05.616779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 339/678] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:05.617863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.628264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.628954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.629238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.629515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.630614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:05.630926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.632994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d5c9-74c0-4d89-934e-dc8d4b58358d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.651842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 340/679] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:05.651842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb3d2-6fd6-44fe-881e-2cb0669b0b27 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.651842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb3d2-6fd6-44fe-881e-2cb0669b0b27 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.651842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb3d2-6fd6-44fe-881e-2cb0669b0b27 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.656775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 341/680] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:05.656775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bb3d2-6fd6-44fe-881e-2cb0669b0b27 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.692190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:05.703440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:05.725308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4c066-153d-4f65-b449-528e5c7dca19 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.760357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.761013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.761326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.761576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.762437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=15924f58d53145ae83b3b4cfbbb02ef7) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:05.762745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.764384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.779910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 340/681] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/role_assignments?user.id=15924f58d53145ae83b3b4cfbbb02ef7&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:05.779910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.794103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bb3d2-6fd6-44fe-881e-2cb0669b0b27 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.844529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.845231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.845472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.845744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.846586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:05.846887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.848499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7b3c3-1850-47d0-8430-83700afcadbd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.869650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 342/682] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:05.871078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.871440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.871714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.894705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:05.896303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:05.901805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01794f4-f3d5-42e9-8828-1a77dd6617ae None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:05.941985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:05.942853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:05.943285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:05.943568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:05.951919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:05.953471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:05.969345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:05.970275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:05.971150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:05.971574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 341/683] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:05.972225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.977102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 343/684] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:05.978106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:05.983389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.016021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.105463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed85e-a548-4712-b26c-095885afafaf None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.106596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d62f1-e00f-467f-bee8-4aa5f8f26d18 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.141021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.141825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.142194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.142557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.143548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:06.144142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.145622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.149552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.150375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.150860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.151279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.158859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 344/685] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:06.164793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.235778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.237474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.246796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.257795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.281863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d40c2-d208-484c-9cb0-7b1d004e7689 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.329691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.331109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.331394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.331670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.348302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.350467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.356921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 342/686] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:06.358186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:06.375492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:06.376511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:06.377380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:06.377380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:06.377380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:06.377380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:06.377599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 345/687] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:06.379975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.466637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72d50-098c-44a7-9a59-4ff621cc39de None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.474403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.503817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.504722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.505102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.505474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.506518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=7b086db83a6b47fb98d450bb7e1c8026) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:06.507062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.508707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.509805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a226967-d7fb-4e08-a731-1dd7855b658d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.515733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 343/688] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:06 2022] GET /identity/v3/role_assignments?user.id=7b086db83a6b47fb98d450bb7e1c8026&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:06.517031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.574577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.588661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.589497 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.589778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.590056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.612826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.614252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.634921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24533417-245d-45cf-854a-bb8f88d3001e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:06.651003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:06.653137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:06.655141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:06.655141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:06.655141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:06.655141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:06.655141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 346/689] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:06.672966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.678745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.679495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.679781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.680114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.680856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:06.681165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.682557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.695664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 344/690] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:06.701193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.734781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:06.789221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a9cc-ebe1-435f-b3a3-9d367326d6b2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.811347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e44c2e-c324-481e-be4c-204391bce142 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:06.825086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.825764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.826039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.826308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.827035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:06.827337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.828774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.843560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 347/691] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:06.854245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:06.855035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:06.855775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:06.856097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:06.856378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:06.857222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:06.858797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:06.892965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 345/692] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:48:06 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:48:06.909384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:07.003798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fc84e-986d-4f9c-abe8-9098797516c2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:07.058135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:07.058967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:07.059247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:07.059522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:07.077462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:07.078982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:07.100612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:07.102877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:07.104972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 348/693] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:07.108172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:07.212087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:07.227996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450be9c5-0b6a-4ef4-931a-23d71f1790ef None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:07.262965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:07.263671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:07.263991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:07.264280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:07.265094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:07.265413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:07.266969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac86b-748c-43c8-8e4e-c4161734c657 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:07.281595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 346/694] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:07.287149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:07.398040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7bdcf-c5a3-4e42-b5a3-28285b489390 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:07.400720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:07.433036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:07.433698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:07.433973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:07.434240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:07.435074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:07.435375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:07.436845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a05123-c30a-4229-92fa-5653de853dfa None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:07.443340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 349/695] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:07 2022] GET /identity/v3/role_assignments?user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:07.549661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de6d4-2119-4d4c-8428-74fd209fd42b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:07.550039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de6d4-2119-4d4c-8428-74fd209fd42b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:07.550199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de6d4-2119-4d4c-8428-74fd209fd42b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:07.551612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 347/696] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:07.559156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:07.559447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:07.559698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:07.581001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:07.582213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:07.663461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 350/697] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:07.702882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57de6d4-2119-4d4c-8428-74fd209fd42b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:07.827617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57de6d4-2119-4d4c-8428-74fd209fd42b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:07.867358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:07.868491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:07.868911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:07.869335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:07.884352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:07.892086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:07.893207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:07.894262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:07.896677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 348/698] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:07.908943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.020546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b99f-e0fb-4e82-9db3-6d2a65b6eb78 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.022192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:08.063387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.064093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.064374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.064643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.065392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:08.065698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.067069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a328a-c33a-46b3-b73f-6a3a3a238765 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.076949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 351/699] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:08.082253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.096937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:08.102137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaaed4-f7b8-4dde-a070-d97963857b74 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.102421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaaed4-f7b8-4dde-a070-d97963857b74 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.103014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaaed4-f7b8-4dde-a070-d97963857b74 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.104349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 352/700] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:08.111025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.111311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.111580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.147090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:08.148437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:08.238624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7f291-34d4-46d0-8aa4-c124b51b0444 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.274303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.275182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.275466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.275736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.287673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.289504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:08.310175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:08.311075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:08.312002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:08.312002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:08.312002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:08.312002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:08.319892 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 353/702] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 09:48:08.320197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 349/702] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:08.320468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.342067 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.405933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:08.450612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726d17f-15eb-464e-ad5a-f89601e52946 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.485943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.487162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.487635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.488131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.488980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.490800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:08.505502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:08.506493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:08.507445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:08.510434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 350/703] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:08.515924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:08.517615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.529061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad770ace-2b6b-4a2e-8236-447d3ffc840e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.598307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.598910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.599196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.599465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.600232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:08.600536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.602181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.627917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 354/704] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:08.631272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141fd31-5c2e-4f52-b62f-a30d50df0d67 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.641322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.660620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:08.670435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.671401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.671917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.672384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.673291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:08.673804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.675389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.682474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 351/705] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:08.688193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.796435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be2e18-ea0c-448f-819b-615054bee553 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.803997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e553b-cf63-4384-957e-18c4fce531b2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:08.833899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.835006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.835722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.836632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.837629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:08.838127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.839707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.846430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 352/706] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:08.868924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:08.869677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.869955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.870224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.879399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:08.880916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-344ff72b-ae79-459e-bf56-910df13c4268 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:08.903341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:08.904368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:08.905289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:08.912896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 355/707] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:08.919065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:08.975903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9f99d-3fe3-452c-8858-ba802602565a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.975903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9f99d-3fe3-452c-8858-ba802602565a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.975903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9f99d-3fe3-452c-8858-ba802602565a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.975903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 356/708] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:08.977615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:08.977902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:08.978208 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:08.989795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:09.013114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:09.014356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:09.028258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:09.051095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba97ef-9b93-467b-88c7-18692981d7ab None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:09.088044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:09.088781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.089068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.089343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.090194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:09.090506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:09.091973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:09.103200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 357/709] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:09.115278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 353/710] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:09.116656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:09.125235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:09.235637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413cf53-8e26-4e8c-91e2-eaece0bc8b45 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:09.252241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:09.273047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:09.274036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.274409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.274777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] PATH_INFO: `/v3/services/network` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.275590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:09.277201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:09.293569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:09.294512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:09.295334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:09.298307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 354/711] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:09 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:09.304187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:09.306440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cde17-b8a8-46f2-90d5-ddb524bbf0e5 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:09.396853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:09.397481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.397832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.398177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.408959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675c446-2ee1-4e5e-8b0d-ad2ae13a996b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:09.422475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:09.447416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:09.448552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.449064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.449529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.450429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] RBAC: Adding query filter params (name=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:09.451018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:09.452726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:09.458008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 355/712] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:09.464319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:09.532274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:09.568021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9350194-2c28-4a11-b915-deaf88238e01 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:09.604691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:09.605849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:09.606719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.608444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:09.608941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.609405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.610308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] RBAC: Adding query filter params (type=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:09.612598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:09.614189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea3395-f927-49ed-bf5e-9c681d3e6000 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:09.628463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 356/713] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:09.805117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b294b-456b-491a-9655-50b9574fcd25 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.806134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b294b-456b-491a-9655-50b9574fcd25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.806646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b294b-456b-491a-9655-50b9574fcd25 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.808524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 357/714] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:09.810736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 358/715] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:09.814261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.814784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.815087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.815496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:09.842190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:09.843389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:09.868000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:09.937733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c0871-28d0-4384-b592-04fe418c6006 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:09.980736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:09.981544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:09.981826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:09.982096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:09.999508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 358/716] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:10.008990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:10.010468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:10.033589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:10.034546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:10.035432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:10.035432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:10.035432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:10.035432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:10.036396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 359/717] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:10.042917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.077117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.096287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.192501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.202512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d4394-e28e-4a2b-9b6b-073880c5c17f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.256642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c07c73-7e16-42e6-a1bb-2f0646c2db3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.257150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b97a9-e934-4232-bb04-09479c724724 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.257725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07c73-7e16-42e6-a1bb-2f0646c2db3b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.258173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07c73-7e16-42e6-a1bb-2f0646c2db3b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.258457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07c73-7e16-42e6-a1bb-2f0646c2db3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.305700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c07c73-7e16-42e6-a1bb-2f0646c2db3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:10.308146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.308818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.309099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.309376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.310114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:10.310422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:10.311802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575e99-d38a-4193-8ee3-b3401bf0bd27 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-89c07c73-7e16-42e6-a1bb-2f0646c2db3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:10.313457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:10.314429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:10.315282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 360/718] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:10.316735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f281f-e5b6-4beb-9614-c96345fc8e79 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.316735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f281f-e5b6-4beb-9614-c96345fc8e79 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.317005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f281f-e5b6-4beb-9614-c96345fc8e79 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.319352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 361/719] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:10.321157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f281f-e5b6-4beb-9614-c96345fc8e79 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.324582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 359/720] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:10.326878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.327223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.327444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.347472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:10.349147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:10.432869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f281f-e5b6-4beb-9614-c96345fc8e79 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.437434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 360/721] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:10.438313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.473212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.486383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.487155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.487444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.487715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.488495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:10.488801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:10.490176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:10.499492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 362/722] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:10.500458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.517436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.557696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e44e1-3289-42e7-9dca-7ea721474518 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.593900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.594757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.595043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.595317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.605801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:10.607525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:10.607981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4520f-141d-4c69-8c09-a3043c84454f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:10.624982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:10.626705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 361/723] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:10.628745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.644294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.645285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.645708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.646074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.657470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:10.665749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:10.666869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:10.667944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:10.674086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 363/724] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:10.674869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.728926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.758201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.760138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df3c61-568e-4184-a561-c1f1b5c0fb33 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.785561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116a905-6a85-40dd-aafe-1377dbdc77e9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.805823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.807254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.808126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.808550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.822800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.823015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:10.823628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.824027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.824367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.825154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:10.825292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:10.825667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:10.827175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:10.842344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 364/725] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:10.843357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:10.846531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:10.847566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:10.848685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:10.848685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:10.848685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:10.848685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:10.851504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 362/726] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:10.852520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:10.959797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be605-537b-485b-aadf-f2f391f45e10 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.964183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e214b-2cb2-4d06-9fef-25c8b3036806 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:10.990031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:10.996552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:10.997335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:10.997895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:10.998111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:10.999094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=14dd1de1fe914f319017818e571732de) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:10.999497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.001169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.003380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.004353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.004814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.005256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.006175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:11.006651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.007761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 363/727] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/role_assignments?user.id=14dd1de1fe914f319017818e571732de&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:11.008674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.008781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.019018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 365/728] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:11.019303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.025235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.122535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192b911-b34b-49b8-94e5-fa0bccbcf85c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.143533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345172b-652c-41ed-9c8f-62cd87b07e46 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.168766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.169707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.169707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.170002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.170721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:11.171018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.175910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.191900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 364/729] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:11.193161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.212309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.213629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.214046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.214537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.235909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.237817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:11.261884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:11.262865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:11.263727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:11.263727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:11.263727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:11.263727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:11.267953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 366/730] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:11.269765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.274776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.325910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9317c-1dfc-4b37-a039-4465bf39104a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.357824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.395320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86937-76e0-4689-b758-485592287be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.396414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86937-76e0-4689-b758-485592287be4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.396833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86937-76e0-4689-b758-485592287be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.397244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b86937-76e0-4689-b758-485592287be4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.412916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b86937-76e0-4689-b758-485592287be4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.415863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b86937-76e0-4689-b758-485592287be4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-92b86937-76e0-4689-b758-485592287be4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:11.436571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:11.437758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:11.438886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:11.445250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 365/731] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:11.449694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86937-76e0-4689-b758-485592287be4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.458693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e3d4-b284-4bfe-aa26-259b9b52402a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.499563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.500275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.500553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.500827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.502027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:11.502349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.504202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44dfd3-a9c3-45de-98a3-aa8ec1a69863 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.518686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 367/732] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:11.520445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9fa2-1151-4e1f-940a-0b4aec9d4c64 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.520925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9fa2-1151-4e1f-940a-0b4aec9d4c64 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.521403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9fa2-1151-4e1f-940a-0b4aec9d4c64 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.523112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 368/733] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:11.524609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9fa2-1151-4e1f-940a-0b4aec9d4c64 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.591189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b86937-76e0-4689-b758-485592287be4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.616564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.641373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.642316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.642601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.642879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.644379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:11.644694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.646483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca7d81-68bd-4c33-83b9-e245053845b4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.647862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9fa2-1151-4e1f-940a-0b4aec9d4c64 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.659308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.671345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 366/734] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:11.673299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.673541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.673817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.689432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.690418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.690818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.691192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.693683 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:11.695505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:11.700125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.701765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:11.720659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:11.722446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:11.724573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:11.731890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 369/735] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:11.731890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.774311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 367/736] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:11.775247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.856651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045b79b-6973-4f60-88a3-c24af559cbf7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.883221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a58200-273c-4018-9d39-f48d16705d93 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:11.885768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.892432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595a0c7-4a78-42c4-bd72-3bc9600c0fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.893092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595a0c7-4a78-42c4-bd72-3bc9600c0fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.893377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595a0c7-4a78-42c4-bd72-3bc9600c0fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.893652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595a0c7-4a78-42c4-bd72-3bc9600c0fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.919835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:11.920460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:11.920739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:11.921015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:11.921857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:11.922164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.923601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:11.934799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:11.941880 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 368/737] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:11.942528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:11.976358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595a0c7-4a78-42c4-bd72-3bc9600c0fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:11.977848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595a0c7-4a78-42c4-bd72-3bc9600c0fd1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.051794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c391e2-67b3-4e5d-9625-36e146296ce3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.083226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 370/738] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:12.084666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37093578-c56c-43db-94e0-543ed24abcb8 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.084973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37093578-c56c-43db-94e0-543ed24abcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.085267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37093578-c56c-43db-94e0-543ed24abcb8 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.087248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 371/739] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:12.088090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37093578-c56c-43db-94e0-543ed24abcb8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.094019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.094791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.095138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.095484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.096328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.097802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.123903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 369/740] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:11 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:12.123903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.150895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:12.181323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:12.246070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37093578-c56c-43db-94e0-543ed24abcb8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.246443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0dcdf-158f-444d-b325-78b10b801286 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.298650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.299619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.299932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.300212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.301764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.302818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.303391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.303896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.313013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.314489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-25719a66-3177-45af-a7f1-e0618e62d371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:12.332825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:12.333771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:12.334985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:12.334985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:12.334985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:12.334985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:12.334985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 370/741] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:12.335621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.335941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.336540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.353472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:12.354662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:12.386428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.388019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.430511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 371/742] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 09:48:12.440668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.440668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:12.480066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:12.530017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 372/743] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 09:48:12.531353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.565962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293521-1137-462f-9687-59f6d6d97c34 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.602028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.602705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.603045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.603325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.604254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:12.604430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.605801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.618441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 372/744] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:12.619286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.645377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2e575-199b-4615-9009-ffd89c90f0ec None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.684856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.685888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.686357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.686911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.687611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:12.689032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.690621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.716377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 373/745] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:12 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:12.717747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.731948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f33090-52d6-43d4-85e2-0c47102769d0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.764263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:12.776663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.777572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.777846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.778154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.793227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.794854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:12.813853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:12.814859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:12.815740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:12.815740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:12.815740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:12.815740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:12.815740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 373/746] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:12.825485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:12.876451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1670d8-fb98-46d8-bf22-025df828f67b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:12.914435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:12.915757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:12.916269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:12.916736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:12.933806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:12.936469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:12.959400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:12.960735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:12.961991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 374/747] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:12.965317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:13.016768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dfee9-4156-47e4-b412-973f98a9a29d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.040326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:13.068805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.070080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.070335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:13.071317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.071828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.073236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:13.073743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:13.075534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:13.090290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d92c5-d3bf-4881-b625-fd2b3a84763b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.094003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 374/748] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:13.102798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:13.132639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.133390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.133718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.133998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.134895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:13.135207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:13.137531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:13.153278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 375/749] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:13.159137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:13.242492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc663bc8-d194-4a8d-b67b-5be7b7ce58ed None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.284302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.285295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.285576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.285875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.294507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:13.295996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:13.311073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75565-b413-4abd-a102-2b1aded904a6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9c1f01fa-c1d5-414d-8189-25357df1e64d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:13.314701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:13.315559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:13.316905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 375/750] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:13.317569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164fc74-3249-46ae-93de-5b13e3f05ede None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.318678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164fc74-3249-46ae-93de-5b13e3f05ede None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.318678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164fc74-3249-46ae-93de-5b13e3f05ede None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.320189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 376/751] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:13.324817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164fc74-3249-46ae-93de-5b13e3f05ede None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:13.352083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.352793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.353415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.353834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.354909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=cc7643dc9f834584973cb7fdedbb4d70) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:13.355364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:13.357320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:13.358242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1b93-a48c-4397-8c46-575811ac93bd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:13.365362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 376/752] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:13 2022] GET /identity/v3/role_assignments?user.id=cc7643dc9f834584973cb7fdedbb4d70&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:13.370610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18013d69-c462-4792-9164-39e54850deda None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.370903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18013d69-c462-4792-9164-39e54850deda None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.371162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18013d69-c462-4792-9164-39e54850deda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.391565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-18013d69-c462-4792-9164-39e54850deda None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:13.393923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-18013d69-c462-4792-9164-39e54850deda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:13.413186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:13.507188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164fc74-3249-46ae-93de-5b13e3f05ede None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.533686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 377/753] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:13.555620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.556708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.557189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.557660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.558656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:13.559157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:13.560848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:13.567094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18013d69-c462-4792-9164-39e54850deda None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:13.578418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 377/754] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:13.584213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:13.697418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:13.706753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a951b47-9201-4fb3-8dab-467cda421b8f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.743245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.744327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.744811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.745307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.746344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:13.746866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:13.748546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:13.755098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 378/755] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:13 2022] GET /identity/v3/role_assignments?user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:13.805551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18013d69-c462-4792-9164-39e54850deda None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:13.866054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:13.866970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:13.867255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:13.867535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:13.875694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:13.884731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:13.898905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:13.899981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:13.900828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 378/756] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:13.906500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.047438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cfd7-4e87-4c06-98dd-d7f053dc7009 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:14.083451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:14.084558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.085140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.085828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.087218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:14.087756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:14.089355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:14.098737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 379/757] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:14.104360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.268114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81551d78-36df-4d78-8223-a6f0db49a6e3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:14.329760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:14.330940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.331430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.331948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.351634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:14.353472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-05bf8efa-4b21-430f-93d5-88e1630b7cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:14.376308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:14.377255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:14.378369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:14.378369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:14.378369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:14.378369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:14.378369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 379/758] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:14.383678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.403625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:14.488507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16c8b7-cd61-4662-827f-eeb47ec6d61e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:14.504404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394280-2db9-4607-bf66-28373456d847 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.505309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394280-2db9-4607-bf66-28373456d847 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.505834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394280-2db9-4607-bf66-28373456d847 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.508243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 380/759] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:14.513404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.513915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.514381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.535843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:14.536390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:14.536506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.536742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.537018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.537769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:14.538128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:14.539064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:14.539579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:14.556439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 380/760] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:14.562082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.604320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:14.657953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 381/761] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:14.698685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:14.705180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.743351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750532e-afbe-4024-843b-9ff276a27df4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:14.838328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fe459-e67d-4f27-a907-e7f203d99271 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:14.842425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:14.843248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.843534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.843815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.863903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:14.865412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:14.874672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:14.875769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:14.876243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:14.876699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:14.877503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:14.879041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:14.894055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:14.895225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:14.896292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:14.897255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:14.898149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:14.899054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:14.900927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 382/763] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jun 30 09:48:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:14.905632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 381/763] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:14.906667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.908832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:14.954743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.012013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed833-9200-49a8-a01e-5fd17b90ec5d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.047320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.048329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.048756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.049174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.050153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:15.050615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.052409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.071620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 383/764] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:15.079355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.118271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac31c2-a7e9-453d-9d0f-cff6dc5a2e23 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.178482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.179392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.179673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.179976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.180704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:15.181012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.182637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.191850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 382/765] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 09:48:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:15.197750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.206886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.236745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03952cd2-5f01-487f-9124-418c72750fb4 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.239062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.279673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.280874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.281351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.281818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.297222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.298999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:15.319102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:15.320237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:15.321227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:15.321227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:15.321227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:15.321227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:15.323652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 384/766] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:15.333101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.389222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b598a-90bc-4ce4-bf8e-02ac284a06dd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.435009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.436261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.436716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.437373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.438633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:15.439086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.440645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef8f5-ca9b-4f62-85f1-31dd63590c6a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.448429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 383/767] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:15.453231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfdcf0-54d3-4872-ad0a-977af24275cb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.453231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfdcf0-54d3-4872-ad0a-977af24275cb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.453231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfdcf0-54d3-4872-ad0a-977af24275cb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.455948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 384/768] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:15.457085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfdcf0-54d3-4872-ad0a-977af24275cb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.462390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ebca-6669-44a2-a90a-de7811f2233a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.498288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.499335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.499844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.500376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.501299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:15.501803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.503380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72d57f-a54e-45dd-af1b-58daa3063793 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.515998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 385/769] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:15.517667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.518162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.518634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.536020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:15.537571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:15.548318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.624053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 386/770] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:15.625514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.676767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.706953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfdcf0-54d3-4872-ad0a-977af24275cb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.742233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.742908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.743190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.743464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.744354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] RBAC: Adding query filter params (interface=public, service_id=42f01cfca42d4cf18750cb3e4d2d7b50, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:15.744667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.746120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.758802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 385/771] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/endpoints?service_id=42f01cfca42d4cf18750cb3e4d2d7b50&interface=public®ion_id=RegionOne => generated 194 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:15.758802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.766255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554bdb5-5221-4f7d-9af0-7738852c790f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.801761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.802575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.802859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.803135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.811150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.812666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-8b78cd06-6cc1-444e-9d33-88174e7cf79e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:15.827053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:15.827976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:15.828873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 387/772] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:15.829804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d899f6-f4ee-4f3a-b2e4-8d559302f93a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.829898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d899f6-f4ee-4f3a-b2e4-8d559302f93a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.830767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d899f6-f4ee-4f3a-b2e4-8d559302f93a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.835645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 388/773] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:15.835645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d899f6-f4ee-4f3a-b2e4-8d559302f93a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.841514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.868637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6686c-ffb7-47d7-8f4b-ac55c312e53a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.899927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:15.906838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.907637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.907937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.908243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] PATH_INFO: `/v3/services/network` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.908885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.910252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:15.921855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:15.922861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:15.923718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 386/774] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:15.924547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:15.956619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d899f6-f4ee-4f3a-b2e4-8d559302f93a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:15.992717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b9b48-b0b2-4d30-927a-6cb00b329d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:15.993395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b9b48-b0b2-4d30-927a-6cb00b329d80 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:15.993679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b9b48-b0b2-4d30-927a-6cb00b329d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:15.993956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b9b48-b0b2-4d30-927a-6cb00b329d80 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:15.994649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b9b48-b0b2-4d30-927a-6cb00b329d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:15.996067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b9b48-b0b2-4d30-927a-6cb00b329d80 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.001545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 389/775] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:16.002807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.003102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.003378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.031419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:16.033343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:16.058690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7ef48-6c5a-45d8-a753-b6e82f7d099a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.106194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.107611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.108662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.108956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.110115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.110115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:16.110349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.111924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.113247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 390/776] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:16.114587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.127707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 387/777] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:16.133774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.201388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.233883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4ae56-90f2-40bd-a361-711e51e159dc None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.257687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd1a8-680f-4a7c-9990-c20f5df28fef None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.282602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.283318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.283604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.284037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.285058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] RBAC: Adding query filter params (name=network) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:16.285376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.287068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.294824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 391/778] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:16.299693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.304482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c986d88-adc2-4191-bc9a-73d6687174f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.305390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c986d88-adc2-4191-bc9a-73d6687174f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.305672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c986d88-adc2-4191-bc9a-73d6687174f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.305950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c986d88-adc2-4191-bc9a-73d6687174f0 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/14dd1de1fe914f319017818e571732de/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.308044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c986d88-adc2-4191-bc9a-73d6687174f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=14dd1de1fe914f319017818e571732de, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.309493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c986d88-adc2-4191-bc9a-73d6687174f0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.326726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 388/779] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:16 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/14dd1de1fe914f319017818e571732de/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 09:48:16.328349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84902486-e8cf-497a-8aa0-e72d611ff879 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.328939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84902486-e8cf-497a-8aa0-e72d611ff879 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.329252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84902486-e8cf-497a-8aa0-e72d611ff879 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.331233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 389/780] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:16.332318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84902486-e8cf-497a-8aa0-e72d611ff879 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.369214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.411857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d9547-39c4-4938-a3bb-963c6dbbab2a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.459510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491de30c-c77c-4df8-b340-93d9e48e2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.460092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84902486-e8cf-497a-8aa0-e72d611ff879 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.461492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491de30c-c77c-4df8-b340-93d9e48e2144 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.462014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491de30c-c77c-4df8-b340-93d9e48e2144 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.462811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491de30c-c77c-4df8-b340-93d9e48e2144 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.478950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491de30c-c77c-4df8-b340-93d9e48e2144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:16.480158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-491de30c-c77c-4df8-b340-93d9e48e2144 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:16.492214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:16.493909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:16.494808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:16.498418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 392/781] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:16.500777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.501227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.501639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.510497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.511340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.511760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.512824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.513632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] RBAC: Adding query filter params (type=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:16.514011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.515479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.522353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 390/782] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:16.523290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.523528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:16.524948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:16.606376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 393/783] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:16.607247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.652472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ba4d8-26aa-4d25-88a1-e19cc720656a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.683016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.689584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.690381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.690758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.691105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.691969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:16.692344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.693806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.703083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 391/784] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:16.704194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.733098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9e0fc-e53d-40ec-a7d7-051d90eb712e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.747139 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.768807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.769484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.769768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.770040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.771078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] RBAC: Adding query filter params (interface=public, service_id=96f08906a62a46e1809847e4f83aab4f, region_id=RegionOne) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:16.771422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.772831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.780771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 394/785] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/endpoints?service_id=96f08906a62a46e1809847e4f83aab4f&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:16.781422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.811906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c10c7-8082-446b-a657-0e265ed56494 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.855018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.856936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.857506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.857965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.869851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:16.877700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:16.878629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:16.879464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:16.884235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 392/786] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:16.885256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:16.899269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c865fd5-027a-4515-b099-f07208ab8591 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:16.921608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:16.945734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:16.947503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:16.948097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:16.948798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:16.966334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:16.968915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:16.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:16.995387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:16.996979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:16.996979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:16.996979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:16.996979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:16.996979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 395/787] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:16.996979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.005919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:17.012367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df085e9e-fb68-48a3-b093-8fc66ec1f4c2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.062870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.064035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.064525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.065021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.065914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.067554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.074508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 393/788] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:17.075717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.151301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10abd12-591e-4ad9-845b-b4f7e6b83641 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.198978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd28072-237d-40b5-997b-910da1e102d4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.224912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.225786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.226062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.226339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.227253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:17.227563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.229156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.240461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.241625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.242202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.242693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.243774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:17.244386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.246108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.252846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 396/789] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:17.258993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.262496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 394/790] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:17.267823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.324665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:17.341618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:17.431572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0d0a7-23a4-4fb4-98a6-7e52dc6c70ad None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.438636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784cf41-1332-4e37-b2e2-bfc33a78dd85 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.472594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.473873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.474394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.474980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.482137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.482962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.483241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.483571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.491317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.492601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.493443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.494082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:17.511720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:17.512757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:17.514285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 395/791] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:17.515787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:17.516746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:17.518031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:17.518031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:17.518031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:17.518031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:17.518591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.520331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 397/792] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:17.534759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.623988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:17.657047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb71ce9-6043-499d-bacb-8400d37e0915 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.692270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:17.724595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.727856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.728164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.728727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9569bd-bd32-4074-95ac-c372d5d2f420 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.730436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.731307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:17.731624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.733118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.770442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 396/793] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:17.776088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.796776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:17.798280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:17.798978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:17.799444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:17.800602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:17.801359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:17.802950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:17.827076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 398/794] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:17.830431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:17.958804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c91bb-5ad4-4f3e-a837-88e2579bbb5b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:17.996130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33439-cd3f-4cd2-8da9-c31b588451ab None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.022617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.023347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.023627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.023916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.044879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.046367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.052171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.055391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.056232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.056516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.056789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.070190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.071667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:18.078591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:18.079521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:18.080470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:18.080470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:18.080470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:18.080470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:18.081120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 397/795] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:18.087280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:18.104114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:18.105099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:18.106009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:18.113366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 399/796] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:18.117837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.133660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.258941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83959-7eaf-4467-900f-2317a1795797 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.272928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d55db-0aa6-46dd-980f-6c81b9d8b593 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.312424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.313103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.313387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.313663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.313848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.314505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:18.314815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.315125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.315586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.316068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.316292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04852398-0604-4406-8d3e-ec7c9bfafa90 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.317239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:18.317977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.320549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.339770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 400/797] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:18.340189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 398/798] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:18.341238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372a3b-8931-4a5f-90df-e6f8e899492c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.341352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.341523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372a3b-8931-4a5f-90df-e6f8e899492c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.341802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372a3b-8931-4a5f-90df-e6f8e899492c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.343447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 401/799] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:18.346711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372a3b-8931-4a5f-90df-e6f8e899492c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.408250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.422822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.459979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5879d-592d-4d64-8d46-33b8c90d73ca None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.477367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372a3b-8931-4a5f-90df-e6f8e899492c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.501816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.502902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.503323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.503729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.513199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.515522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.519340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.520307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.520654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.521042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-faffc4ee-f56a-4c8a-b00b-58ecfed582fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:18.533415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:18.534516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:18.535625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:18.536177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.536834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.543105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 399/800] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:18.543105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.543105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.543105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:18.561433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:18.562366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:18.563242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:18.563242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:18.563242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:18.563242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:18.565632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:18.567047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:18.567475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 402/801] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:18.568458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.642958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 400/802] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:18.644137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.679505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434830-8606-4ac3-a2fa-fdb46f5824e6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.693047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.701029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.717780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.718443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.718721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.718987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.719774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:18.720118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.721558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09455305-9735-4f11-97a3-f44fda51d186 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.733533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 403/803] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:18.734561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a7613-ff92-45e0-bebc-ef11b7092eac None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.734848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a7613-ff92-45e0-bebc-ef11b7092eac None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.735119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a7613-ff92-45e0-bebc-ef11b7092eac None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.740310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 404/804] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:18.740310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a7613-ff92-45e0-bebc-ef11b7092eac None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.763831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d44ed-b377-46ff-a634-7f91f452f092 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.799717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.800356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.800780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.801066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.802112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:18.802679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.804447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.820026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 401/805] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:18.820962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:18.850594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a7613-ff92-45e0-bebc-ef11b7092eac None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.884849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.885750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.886031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.886301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.886955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.888353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4c986aaa-5ae5-465e-aae6-ace93931bd7a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:18.902793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:18.903735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:18.904622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 405/806] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:18.906152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.906152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.906152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.922240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:18.923409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:18.929825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1037735-82b0-404e-a6bd-fcc0a3f6b973 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:18.932857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.955259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:18.967059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:18.968003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:18.968284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:18.968556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/cc7643dc9f834584973cb7fdedbb4d70/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:18.983240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=cc7643dc9f834584973cb7fdedbb4d70, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:18.984740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.011355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 406/807] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:19.011607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 402/808] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:18 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/cc7643dc9f834584973cb7fdedbb4d70/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:48:19.011696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.011769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.144451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c44ff6-3359-4ea1-b47a-be3f1bc77202 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.165878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf433c8d-935e-4873-844b-e2b303757e5e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.197785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.198594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.198980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.199253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.208935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.208935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.208935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.208935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.209306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:19.209306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.209464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.209537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.209604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.214510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 403/809] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:19.215360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:19.228500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:19.229500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 407/810] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:19.230796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.243158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:19.280698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:19.357750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfffa6e-0dbe-4b98-8ef4-800b52ab7e82 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.372163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db2d74-1d80-4958-b437-b368089424dc None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.418332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.419060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.419341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.423900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.436815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:19.445952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:19.448159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:19.450109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:19.456053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 404/811] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:19.456053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.461968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.462227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.462513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.462784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.463516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:19.463822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.467975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.484240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 408/812] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 09:48:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:19.485479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.541498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:19.564008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069a6c1-fe13-4642-9f61-19067fd32c67 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.595167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:19.605051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.605739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.606021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.606296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.606393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae2bd5-9093-40c8-9a7e-cea3e70704c9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.607096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:19.607405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.610040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca74345-e44f-4d3e-b58c-7662c87ec75e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.629535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 405/813] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:19.630942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990cb8d-c2f0-4e66-a4d3-37d47d1d8f25 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.631228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990cb8d-c2f0-4e66-a4d3-37d47d1d8f25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.631736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990cb8d-c2f0-4e66-a4d3-37d47d1d8f25 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.633356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 406/814] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:19.633880 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990cb8d-c2f0-4e66-a4d3-37d47d1d8f25 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.656353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.657091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.657375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.657594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.658456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:19.658656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.660069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.673316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 409/815] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:19.673316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.747065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990cb8d-c2f0-4e66-a4d3-37d47d1d8f25 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.784491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e8885-29dd-4538-a96e-3fba1422a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.785558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8885-29dd-4538-a96e-3fba1422a31c None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.786036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8885-29dd-4538-a96e-3fba1422a31c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.786490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8885-29dd-4538-a96e-3fba1422a31c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.787439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e8885-29dd-4538-a96e-3fba1422a31c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.788536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563549f-186c-4644-b27d-48c209d56ca6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:19.789661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e8885-29dd-4538-a96e-3fba1422a31c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.798551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:19.814601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 407/816] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:19.816394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.816922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.817374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.847100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:19.848706 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:19.861776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:19.862755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:19.863041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:19.863317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/64bf7840b5fd4b3b9dd2b8e18b4495b3/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:19.865427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=64bf7840b5fd4b3b9dd2b8e18b4495b3, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:19.867035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:19.892863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 410/817] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:19 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/64bf7840b5fd4b3b9dd2b8e18b4495b3/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:48:19.895900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:19.921936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:20.013714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 408/818] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:20.044400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.111106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176cb4-751b-4545-ab99-7bb01141dd3d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.154669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.172157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.172157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.172157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.174705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.177454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.197328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1109aef-fdc1-43ba-9288-bad20675c1e3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:20.209380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:20.210454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:20.211769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:20.211769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:20.211769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:20.211769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:20.222451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 411/819] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:20.229690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.249686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:20.271988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.271988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.271988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.271988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.272592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.272592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:20.293331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:20.294355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:20.295973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 409/820] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/services/network => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:20.303932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.331505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:20.371130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb912d-345e-4629-a8cd-b9da8a9ae304 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.413643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.414317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.414581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.414851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.415580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:20.415900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.417291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.440354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 412/821] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:20.446000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.560558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f56016-5516-4d79-9cd6-2f21f6cd23f4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.599169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.600319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.601326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.601813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.602810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:20.603325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.605242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.611188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 410/822] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:20.614664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f19c6a-364a-4c28-9e79-e799d1551a46 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.616526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.668721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:20.687174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.690504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.691047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.691530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.707998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.709901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:20.726057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:20.727065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:20.728071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:20.729569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:20.735975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 413/823] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:20.737654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.841335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdced8-73a7-400c-9b49-58d627d3d5f3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.871901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc9fba-79ee-4291-99bd-df042f0008b5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:20.888137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.889179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.889659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.890142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.891190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:20.892312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.893926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.932492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 414/824] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:20.935787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.948649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:20.949588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:20.949588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:20.949909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:20.950573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] RBAC: Adding query filter params (type=network) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:20.950881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:20.952288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:20.974365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 411/825] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:20.979554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:20.984663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.055056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18b9e2-b526-4c2a-a2ee-200351346da6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.143495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.144404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a01922-507f-42a9-9df9-4cdad3b5c648 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.145279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.147306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.147844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.170583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.208612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210005a-83c2-42f6-9e46-d215a187a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.209319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210005a-83c2-42f6-9e46-d215a187a147 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.209822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210005a-83c2-42f6-9e46-d215a187a147 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.210476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210005a-83c2-42f6-9e46-d215a187a147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.211572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210005a-83c2-42f6-9e46-d215a187a147 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:21.213006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210005a-83c2-42f6-9e46-d215a187a147 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:21.224458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:21.226654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:21.261844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 412/826] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:48:21.268746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.277018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ff74-362e-47e3-8d49-02f1055221f4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.277373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ff74-362e-47e3-8d49-02f1055221f4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.277581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ff74-362e-47e3-8d49-02f1055221f4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.279503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 413/827] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:21.285106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.287490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.287490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.321126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:21.322447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:21.342473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 415/828] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 09:48:21.348531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.467923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720882cd-4cfe-45e7-b44b-a77441f3d6ec None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.473653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 414/829] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:21.498532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.508883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.510066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.510870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.511339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.520516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.525817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:21.527746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:21.551771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:21.552756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:21.553887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:21.553887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:21.553887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:21.553887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:21.553887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 416/830] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:21.561305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.630578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.637834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed959ed9-ad7e-4d99-91f7-b8ab8aa81578 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.674320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.675129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.675413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.675677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.686390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:21.694146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:21.695631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:48:21.696644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:21.699236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d36fb-113b-472e-a362-773f7150c5e2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.701364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 415/831] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:21.703069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.738429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.739589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.740173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.740678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.741749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:21.742275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:21.743911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:21.759560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 417/832] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:21.765614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.807394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcc276-bc8f-4b2f-8b65-1d7a79b281fb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.821569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.843534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.844240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.844520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.844793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.845544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:21.845931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:21.847564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:21.862066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 416/833] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:21.863330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.881129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17245122-df99-4b19-97b8-6efefcdb6b84 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:21.886284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:21.921425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:21.922711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:21.923223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:21.923757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:21.931716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:21.933634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:21.953158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:21.954112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:21.955001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:21.957602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 418/834] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:21.965593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:21.973787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396ca8-05b2-431b-8218-492f8155c18e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.013910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.014662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.014941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.015214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.026773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.028264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:22.046602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:22.047553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:22.048483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:22.048483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:22.048483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:22.048483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:22.052616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 417/835] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:22.053961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.098506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9899d-e91d-4ffe-8cc6-18cbcfdc188d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.113874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.128205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.143154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.143815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.144122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.144393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.145181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:22.145486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.146919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd54723-4a80-4150-91e7-d876c0603c88 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.163802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 419/836] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:22.164482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86c222-5c9f-42c5-bad3-65635b8a31b5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.164579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b0167-6a38-43ed-b869-8ccc73274644 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.164579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b0167-6a38-43ed-b869-8ccc73274644 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.164579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b0167-6a38-43ed-b869-8ccc73274644 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.164579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 420/837] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:22.166673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b0167-6a38-43ed-b869-8ccc73274644 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.200485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.201150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.201436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.201707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.202442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:22.202750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.204165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112556-a421-4bc9-bdc4-915ec2694c74 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.217088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 418/838] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:22.218191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.218477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.218751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.236656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:22.237849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:22.282264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b0167-6a38-43ed-b869-8ccc73274644 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.316552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 419/839] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:22.317665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.325885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.326790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.327311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.327790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.328848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=14dd1de1fe914f319017818e571732de) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:22.329366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.331120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e34d99-c557-435f-80a4-9083374b82d3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.338637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 421/840] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/role_assignments?user.id=14dd1de1fe914f319017818e571732de&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:22.340396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68578795-9033-47da-9f23-6902df7ef6b9 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.340894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68578795-9033-47da-9f23-6902df7ef6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.341366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68578795-9033-47da-9f23-6902df7ef6b9 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.343134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 422/841] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:22.344266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68578795-9033-47da-9f23-6902df7ef6b9 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.388403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.393986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.436666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88f8d4-ca5b-418f-bc99-32ce283a1a8c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.467773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68578795-9033-47da-9f23-6902df7ef6b9 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.474852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.475941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.476229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.476507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.484748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.486237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8bd52559-0ea4-481e-beb8-7045c65489fd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:22.502601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:22.503579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:22.504603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:22.507979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 420/842] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:22.509307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.509613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.509893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.510858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.511669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.511983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.512259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.512928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.514322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:22.526885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:22.527953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:22.528786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:22.529513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:22.529588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 423/843] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:22.529588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.529725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:22.604012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 421/844] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:22.612897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.677663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.730375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ce874-1010-4884-87ce-0ad0af08eddd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.753659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.758741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54c59d-5e70-4bee-b811-42008f0d21a8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.767523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.768218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.768499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.768772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.769492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:22.769795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.771176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.780219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 422/845] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:22.780219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.799124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.799859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.800167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.800447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.801338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:22.801645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.803077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.821547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 424/846] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:22.822132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.889720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa439e-c0d7-4cd1-aa94-cdc6267393fa None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.906557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:22.927116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.927940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.928223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.928702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.929269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd92965-2ed2-4cac-87f0-c89dad4a2f85 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:22.940007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:22.947551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:22.948666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:22.949508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 423/847] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:22.950254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:22.969675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:22.970351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:22.970640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:22.970915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:22.971631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:22.971977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:22.973357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:22.987898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 425/848] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:22.987898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.016580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.068999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12207346-c924-4c0f-8c07-f3e21b930857 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.110227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.110898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.111182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.111467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.124931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc868e1-0ca1-4599-a4d6-85fd79a9890f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.164725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.176176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.177638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.178203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.178693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.179516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:23.180240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.181446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.192119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 426/849] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:23.198239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.198700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.200221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.305611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.329502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 424/850] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:23.331035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0403a-87f1-460f-910c-b2ffa508d1d8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.336242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.376952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.377792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.378630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.378915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.394665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.396458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:23.419590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:23.420675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:23.421548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:23.421548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:23.421548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:23.421548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:23.422425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.423514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 427/851] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:23.431517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.457542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ed0b1-dbef-47ce-abef-a6ee4bb3b5ea None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.503496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.504350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.504629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.504905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.520541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.522825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.545920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f800c5-5b4f-498c-969b-f8e493ec5506 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:23.547227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:23.548113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:23.549019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:23.549019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:23.549019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:23.549019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:23.549019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 425/852] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:23.555344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.601118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.601804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.602083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.602355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.603487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:23.603794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.605506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.607737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.624059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 428/853] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:23.629190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.665298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa4e06-8311-4e41-a271-d238a53ebb3d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.694135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.704224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.705231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.705702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.706147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.707155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:23.707599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.709264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.727470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 426/854] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:23.728726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.742427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958ac8a-e924-4d2e-bf0f-f7870a5f76d1 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.776347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.777286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.777658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.778021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.786008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.787732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:23.803050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:23.804810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:23.805812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 429/855] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:23.810102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.840567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdaeb9-460c-4426-8cca-6ee31b9c7561 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.870236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.884682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.886095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.886532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.887038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.894766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.896634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:23.911778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:23.912740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:23.913635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:23.920263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea22f2-7fe1-4330-adab-0ae5d0300412 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:23.920850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 427/856] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:23.921909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:23.953955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:23.954585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:23.954907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:23.955179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:23.956248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:23.956556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:23.957984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:23.976794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:23.979920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 430/857] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:23.985415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.040865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87b63f-1695-47be-8a1e-05b08da6187c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:24.079825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:24.080637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.080927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.081210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.082051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:24.082362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:24.083921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:24.097579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 428/858] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:24.104000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.121072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afa94-20fa-4362-a9ba-66020992f373 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:24.135839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:24.165151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dd170-1ff9-4ca3-a7ee-2d8139e47916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:24.165820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dd170-1ff9-4ca3-a7ee-2d8139e47916 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.166101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dd170-1ff9-4ca3-a7ee-2d8139e47916 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.166374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dd170-1ff9-4ca3-a7ee-2d8139e47916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.254619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3912a-2a58-4f6f-a469-220ce6cebefc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:24.257965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dd170-1ff9-4ca3-a7ee-2d8139e47916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:24.259439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dd170-1ff9-4ca3-a7ee-2d8139e47916 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:24.278028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:24.322855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:24.324641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.325150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.325627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.327305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=cc7643dc9f834584973cb7fdedbb4d70) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:24.327840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:24.330408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:24.341574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 429/859] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:24 2022] GET /identity/v3/role_assignments?user.id=cc7643dc9f834584973cb7fdedbb4d70&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:24.404840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 431/860] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 09:48:24.416026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.501939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583047-ebba-4a2a-ac66-92cb89b8cc2d None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.502281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583047-ebba-4a2a-ac66-92cb89b8cc2d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.502545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583047-ebba-4a2a-ac66-92cb89b8cc2d None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.506845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 432/861] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:24.514054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.514054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.514054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.528799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b69c8-0028-44fc-865d-56fdb9ed05b0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:24.536891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:24.538479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:24.547659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:24.580654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:24.581470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.581781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.582090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.599388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:24.601394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:24.625935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:24.626907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:24.627794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:24.627794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:24.627794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:24.627794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:24.630083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 430/862] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:24.637849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.644667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 433/863] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:24.676164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:24.676839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.778800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19974d-1eab-4d57-a755-cfb7d51c4917 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:24.800907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06303aef-1d5b-425f-8cac-c809409a9652 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:24.821461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:24.822730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.823427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.824068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.824977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:24.825422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:24.826951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:24.845644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:24.846104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 431/864] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:24.848041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:24.848041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:24.848041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:24.848041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:24.849370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:24.850508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:24.863137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:24.864236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:24.865075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 434/865] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jun 30 09:48:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:24.870956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:24.880956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:24.954404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:24.972914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ed313-2df7-4cf1-8229-d5bd594d89e7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:25.001599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdac29-951b-4613-98ca-2030cdafbe44 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:25.012359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:25.013493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.013611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.013820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.021146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:25.022653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:25.039564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:25.039770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:25.040798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:25.041675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:25.043164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.043617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.044106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.044993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:25.045467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:25.045698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 432/866] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:25.046993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:25.047547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:25.053666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 435/867] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:25.061780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:25.136114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:25.172022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d51e-7d36-4a70-88ec-5d225a33069e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:25.178528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90bfec-6c0e-410c-847e-b956d388266f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:25.209717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:25.210388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.210659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.211383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.212122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:25.212428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:25.213826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370cb9b-0f81-40ca-8dd0-634a2ef8ad6e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:25.225193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 436/868] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:25.232843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:25.233513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.233800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.234073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.234897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:25.235198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:25.236734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:25.240419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511edf91-f6f7-4ef3-93a6-201107c34a9c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.240734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511edf91-f6f7-4ef3-93a6-201107c34a9c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.241029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511edf91-f6f7-4ef3-93a6-201107c34a9c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.242761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 437/869] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:25.248464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.248796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.249003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.252159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 433/870] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:25.259590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:25.263744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:25.268382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:25.270165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:25.375898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 438/871] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:25.376205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c3d4c-c869-4da6-bcfd-46a6a00f4cdb None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:25.393648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:25.419955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:25.420988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.421326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.421684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.423362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:25.423737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:25.425272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558565c-7b92-40ab-b356-cef7a7af2cac None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:25.432630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 434/872] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:25 2022] GET /identity/v3/role_assignments?user.id=64bf7840b5fd4b3b9dd2b8e18b4495b3&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:25.473639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:25.520855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd0e7f-7133-416d-a9b7-7fc3e6351045 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:25.558933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05910bac-5403-49c3-b75c-682c09f1d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:25.559552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05910bac-5403-49c3-b75c-682c09f1d531 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:25.559803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05910bac-5403-49c3-b75c-682c09f1d531 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:25.560116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05910bac-5403-49c3-b75c-682c09f1d531 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:25.560876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05910bac-5403-49c3-b75c-682c09f1d531 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:25.562293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05910bac-5403-49c3-b75c-682c09f1d531 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:25.581228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 439/873] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:25 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:48:26.610630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3caa2c-76a9-4558-9c5d-f73be995bfbf None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:26.611195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3caa2c-76a9-4558-9c5d-f73be995bfbf None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:26.611195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3caa2c-76a9-4558-9c5d-f73be995bfbf None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:26.612814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 435/874] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:26.618349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:26.618745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:26.618930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:26.636731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:26.637912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:26.712988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 440/875] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:26 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:26.737202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3caa2c-76a9-4558-9c5d-f73be995bfbf None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:26.811325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:26.847948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3caa2c-76a9-4558-9c5d-f73be995bfbf None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:26.882297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:26.883463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:26.883967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:26.884452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:26.894825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:26.903079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:26.904033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:26.904853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:26.907520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 436/876] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 09:48:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:26.913341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:26.964381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.014955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb96d3-eb50-43a8-ba4f-783beaba5a28 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.049795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.050482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.050758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.051031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.051786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:27.052123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.053697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.062684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 441/877] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:48:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:27.068251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.169286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3c749-5ada-4b37-b11c-1890da611245 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.205758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.206960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.207459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.207953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.216274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.220661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.222096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9d6a7a97-4ae4-4edd-8ed6-c86458b38040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:27.240281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:27.241835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:27.242715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:27.242715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:27.242715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:27.242715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:27.242715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 437/878] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:27.247774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.370298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf5d0d-e9fc-4adb-a54e-1f4c577dd9cc None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.383713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.396073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c537-89a1-4d53-aae4-747e6b516681 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.396553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c537-89a1-4d53-aae4-747e6b516681 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.397006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c537-89a1-4d53-aae4-747e6b516681 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.398736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 438/879] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:27.404402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54070bf7-62ce-4dae-8531-5d491ace9e13 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.404849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54070bf7-62ce-4dae-8531-5d491ace9e13 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.405211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54070bf7-62ce-4dae-8531-5d491ace9e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.405312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.405875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.406157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.406434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.407162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:27.407470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.408878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.421398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 442/880] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:27.422859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-54070bf7-62ce-4dae-8531-5d491ace9e13 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:27.424156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-54070bf7-62ce-4dae-8531-5d491ace9e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:27.426678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.515970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 439/881] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:27.535237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bffd7-0855-4e65-84c5-7e48359a454b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.537813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb92c-70bd-4213-beb6-d3efcee44a0b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.538105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb92c-70bd-4213-beb6-d3efcee44a0b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.538380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb92c-70bd-4213-beb6-d3efcee44a0b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.539828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 440/882] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:27.540569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbb92c-70bd-4213-beb6-d3efcee44a0b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.573871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.575012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.575296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.575565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.583688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.585336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-b183700e-1094-45bd-a20f-2358732c35f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:27.600907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:27.601878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:27.602837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 443/883] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:27.603422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.603865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.603977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.621022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:27.622310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:27.648199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.654553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbb92c-70bd-4213-beb6-d3efcee44a0b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.668072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.693080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 444/884] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:27.693289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d80538-9c86-4311-8a3e-4f40c43edf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.693829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d80538-9c86-4311-8a3e-4f40c43edf0e None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.693952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.694112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d80538-9c86-4311-8a3e-4f40c43edf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.694542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d80538-9c86-4311-8a3e-4f40c43edf0e None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.695263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d80538-9c86-4311-8a3e-4f40c43edf0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.696670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d80538-9c86-4311-8a3e-4f40c43edf0e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.714429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 441/885] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:27 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:27.721546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d0ae-1a7d-4031-b720-816ed8722d98 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.721937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d0ae-1a7d-4031-b720-816ed8722d98 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.722256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d0ae-1a7d-4031-b720-816ed8722d98 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.723684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 442/886] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:27.728561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d0ae-1a7d-4031-b720-816ed8722d98 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.821450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67fa3e-2b7f-409e-b6db-79122c9c5818 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.846089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d0ae-1a7d-4031-b720-816ed8722d98 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:27.861830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.862533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.862811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.863087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.863990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:27.864302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.865801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b9a86-780b-4475-9022-4c7154b0bcac None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.887000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 445/887] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:27.887430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.887771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.888013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.888984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:27.889801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:27.890081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:27.890353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:27.891018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:27.892555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:27.905053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:27.906095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:27.906925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:27.907660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:27.907660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:27.907815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 443/888] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:27.907815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:27.924566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.939009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:27.982499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 446/889] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:27.983413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.026571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02ca8-8cc5-4907-b00e-220b37f0a48f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.062353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.063010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.063295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.063907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.094773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a17f5e-e5b8-4cd4-93c9-b14fbf7086d8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.129406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.130074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.130353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.130623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.131446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:28.131787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.133198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.134528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.136011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.142802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 447/890] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:28.142802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.156787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.174302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.249198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43225e2-8eb0-432f-ac52-c7cc5051dbda None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.257953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 444/891] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:27 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:28.259155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.284506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.285313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.285595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.285873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.286550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.290297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:28.303894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:28.304899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:28.311239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 448/892] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:28.312891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.378349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d0764-f3d8-498e-a4a4-b44fa4a7c1f8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.385465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.408378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc279-4bc8-4ff4-a31e-3c4a85ee018a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.410249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.417706 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.418402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.418688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.418964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.419794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:28.420118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.421522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.426669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 445/893] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:28.427459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.457042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.458580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.459072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.460045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.477289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.479916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:28.505334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:28.506363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:28.507285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:28.507285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:28.507285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:28.507285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:28.510425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 449/894] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:28.515846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.562753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1e8bd-4330-4eff-ae57-c4dd79b77e79 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.609130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.609804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.610086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.610407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.611144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:28.611368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.612971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.618042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 446/895] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:28.624978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.650267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab794fd6-5b71-4d7d-a42f-f9fd9f4e45d8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.686322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.694043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.694251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.695447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.695891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.696307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.697261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:28.697917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.699444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.713551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 450/896] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:28.721792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.747250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e93cb-e144-43f6-8252-4513c5417785 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.790750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.791351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.791677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.791992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.793016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:28.793328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.795181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.803320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 447/897] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:28.809099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.851838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35681-5882-4d24-adbf-bcf6f39755ef None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.896754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.897672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.897953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.898223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.907715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.909213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.925712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20722ff-e352-4d7c-ae2d-c91bae7fe180 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:28.926328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:28.927184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:28.928205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:28.930777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 451/898] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:28.936558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.950043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:28.966284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:28.967579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:28.968434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:28.969124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:28.970286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] RBAC: Adding query filter params (interface=public, service_id=69827c8c02ef41788355a32706804f72, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:28.970742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:28.972967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:28.986024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 448/899] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/endpoints?service_id=69827c8c02ef41788355a32706804f72&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:28.987252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:28.998446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:29.063957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dc7af-a738-4df9-b4e6-cc613f1bbefd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:29.105475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:29.106691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:29.107269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:29.107640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:29.108766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:29.109212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:29.111346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:29.124510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb499c-7aa0-440e-a684-1090d9274226 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:29.130864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 452/900] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:29.132020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:29.162428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c260bc3-f536-462a-ac17-7edcc6c7ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:29.163392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c260bc3-f536-462a-ac17-7edcc6c7ba11 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:29.163771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c260bc3-f536-462a-ac17-7edcc6c7ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:29.164166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c260bc3-f536-462a-ac17-7edcc6c7ba11 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:29.165086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c260bc3-f536-462a-ac17-7edcc6c7ba11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:29.166567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c260bc3-f536-462a-ac17-7edcc6c7ba11 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:29.179897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 449/901] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:28 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:48:29.248741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fbc97-7b7b-4b4b-bb26-ba6158ea8055 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:29.265667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:29.298132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:29.298818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:29.299099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:29.299374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:29.300259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] RBAC: Adding query filter params (role.id=f39bdab856724510a905efb68870ac32, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=cc7643dc9f834584973cb7fdedbb4d70) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:29.300581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:29.302063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff05add-99ad-4fb8-89b6-26e7a93deea4 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:29.309719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 453/902] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:29 2022] GET /identity/v3/role_assignments?user.id=cc7643dc9f834584973cb7fdedbb4d70&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=f39bdab856724510a905efb68870ac32 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:29.787444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc8f93-f339-4781-aa95-078760124ce2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:29.787987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc8f93-f339-4781-aa95-078760124ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:29.788097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc8f93-f339-4781-aa95-078760124ce2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:29.789715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 450/903] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:29.794955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:29.795367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:29.795540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:29.812298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:29.813599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:29.887985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 454/904] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:29.904053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc8f93-f339-4781-aa95-078760124ce2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:29.958169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:30.011902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc8f93-f339-4781-aa95-078760124ce2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:30.047484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:30.048253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.048539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.048814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.049478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:30.051014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:30.064076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:30.065017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:30.065850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:30.070421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 451/905] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jun 30 09:48:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:30.071858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:30.169875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90995d2-66fc-48f6-85da-686b371955e1 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:30.172192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:30.204782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:30.205456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.205738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.206010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.206727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:30.207033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:30.208458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:30.218826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 455/906] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:30.220294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:30.328784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd1cf-c0a6-4ba8-aa5f-4e2df7becc9f None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:30.343211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:30.364357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:30.365041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.365316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.365645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.366366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:30.366674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:30.368077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c247a-7d2a-46c1-9d0f-eeaf11a66d71 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:30.373779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 452/907] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:30.379752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:30.488485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd5125-4347-46c5-b4e8-1d9a1c8ade85 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:30.507847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f834a5-2389-45a3-b0a3-618fb54492e1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.508210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f834a5-2389-45a3-b0a3-618fb54492e1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.508436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f834a5-2389-45a3-b0a3-618fb54492e1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.510056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 453/908] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:30.514860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.515148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.515425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.525335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:30.526050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.526330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.526600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.527393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] RBAC: Adding query filter params (interface=public, service_id=4752aff3103f4be682a5460dabe30852, region_id=RegionOne) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:30.527699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:30.529093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:30.533761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:30.534740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 456/909] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:30 2022] GET /identity/v3/endpoints?service_id=4752aff3103f4be682a5460dabe30852&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:30.535465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:30.540887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:30.576514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:30.614604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 454/910] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:30.638559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:30.658995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651966fb-7c1f-4532-b0ce-462c4f8266f7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:30.704533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fcc70-dcc5-46c2-a5b9-9910d46e975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:30.705200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fcc70-dcc5-46c2-a5b9-9910d46e975a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.705478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fcc70-dcc5-46c2-a5b9-9910d46e975a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.705751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fcc70-dcc5-46c2-a5b9-9910d46e975a None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.706442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fcc70-dcc5-46c2-a5b9-9910d46e975a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:30.707821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fcc70-dcc5-46c2-a5b9-9910d46e975a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:30.718731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 457/911] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:30 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:48:30.750204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e366f2-66ef-4a9b-8644-80f9d8b0ae7f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:30.761867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:30.791033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb1e07-3320-4c03-89ba-b031f09cdcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:30.791718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb1e07-3320-4c03-89ba-b031f09cdcb7 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:30.792023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb1e07-3320-4c03-89ba-b031f09cdcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:30.792301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb1e07-3320-4c03-89ba-b031f09cdcb7 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:30.793070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb1e07-3320-4c03-89ba-b031f09cdcb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:30.794497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb1e07-3320-4c03-89ba-b031f09cdcb7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:30.814270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 455/912] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:30 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:31.367290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963f3f7-298b-4e6b-ba9e-0675e682060c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.368307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963f3f7-298b-4e6b-ba9e-0675e682060c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.368801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963f3f7-298b-4e6b-ba9e-0675e682060c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.370667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 458/913] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:31.376204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.376723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.377199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.394785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:31.396358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:31.417695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4b188-9adc-4fc3-9ba9-e1f2cc6ea676 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.418027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4b188-9adc-4fc3-9ba9-e1f2cc6ea676 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.418240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4b188-9adc-4fc3-9ba9-e1f2cc6ea676 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.419957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 459/914] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:31.424857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.425138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.425413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.443150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:31.444498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:31.494510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 456/915] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:31.511699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:31.534373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 460/916] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:31.559686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:31.589409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:31.628567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:31.637084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec584f-4706-4153-8c06-8e7aba212e8f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:31.673053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2689f-f838-47f6-9fa0-dc8887db7263 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:31.680145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:31.680950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.681237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.681967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.682646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:31.684485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:31.700115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:31.701108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:31.707266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 457/917] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:31.708428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:31.718078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:31.718904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.719179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.719449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.732662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:31.743540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:31.744529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:31.750454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 461/918] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 09:48:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:31.751531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:31.832010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cfbb0-92a3-4593-bbfc-085cf3aefa4e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:31.854577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:31.865943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:31.866738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.867079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.867354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.868148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:31.868454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:31.869882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:31.875811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 458/919] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:31.877862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772c785-62c0-423d-9630-ab1abce6bfe0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:31.885257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:31.912377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:31.931825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:31.933107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:31.933724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:31.934354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:31.935633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:31.936117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:31.938344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:31.956930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 462/920] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:48:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:31.958678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:32.028536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9c2d-847c-41ff-92ef-acbd2c831d0b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.066972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.067691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.068020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.068296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.069059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:32.069367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.070755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.082556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 459/921] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 09:48:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:32.083715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:32.108312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c5179-35de-4a14-87e5-a847c8a5feea None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.153862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.155147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.155779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.156215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.170618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.172589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.174372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:32.185688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:32.190900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167019f-485f-4f47-bda6-d7cc6ad0a0b4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:32.198301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:32.199382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:32.200609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:32.200609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:32.200609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:32.200609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:32.207637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 463/922] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:32.209687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:32.245692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.246970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.248252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.248538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.249360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.250900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.279898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 460/923] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:32.339055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313691-d2c7-4ea4-93a4-faf5ffcb5351 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.386056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.386718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.387008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.387284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.388091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:32.388404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.389794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.404211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 464/924] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:32.413258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:32.522224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ecb7f-c982-46b4-bc99-61e120643541 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.557689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.558560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.558848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.559127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.567167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.568726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:32.584065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:32.585122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:32.586375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 461/925] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:48:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:32.593424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:32.653634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:32.699479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5499b-38be-471e-ba1d-c247a2fb7140 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.739016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.739699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.740025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.740303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.741241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:32.741549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.743014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c7b8-b092-4767-8f52-4396a86c5357 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.761878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 465/926] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 09:48:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:32.767662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:32.770765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:32.823977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e41e-0582-4438-8929-e2d49a1c9bdd None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.823977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e41e-0582-4438-8929-e2d49a1c9bdd None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.823977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e41e-0582-4438-8929-e2d49a1c9bdd None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.823977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 466/927] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:32.828315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834f3f-c633-4f42-a049-bbde24cefe92 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.828595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834f3f-c633-4f42-a049-bbde24cefe92 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.828867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834f3f-c633-4f42-a049-bbde24cefe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.850104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f0834f3f-c633-4f42-a049-bbde24cefe92 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:32.851220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f0834f3f-c633-4f42-a049-bbde24cefe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:32.886699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb41b8c-a773-4676-924e-fb43b85482a3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:32.924832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:32.925641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.925920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.926188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.932169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 467/928] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:32.932169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb288696-b5af-4f45-b660-ea8a7202a020 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.932169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb288696-b5af-4f45-b660-ea8a7202a020 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.932169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb288696-b5af-4f45-b660-ea8a7202a020 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.935142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 468/929] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:32.936337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:32.936626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:32.936905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:32.937562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:32.939027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:32.953596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:32.954794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:32.956830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:32.957742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:32.959588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:32.959588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:32.959588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:32.959588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:32.959588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 462/930] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:32.960417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.007338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.019707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.047893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 469/931] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:33.047893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.077723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a08032-d98e-456b-9bfb-6830e52cb7c1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.118427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.119168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.119450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.119727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.120423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.121803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:33.134533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:33.135770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 09:48:33.136918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.140575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 463/932] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jun 30 09:48:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:33.141894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.149629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19ede9-577f-4809-b397-44e79b091f95 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.185638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66097a07-e369-4933-914b-218c91098721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.186408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66097a07-e369-4933-914b-218c91098721 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.186754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66097a07-e369-4933-914b-218c91098721 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.187096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66097a07-e369-4933-914b-218c91098721 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.187914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66097a07-e369-4933-914b-218c91098721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:33.188475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66097a07-e369-4933-914b-218c91098721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.189666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66097a07-e369-4933-914b-218c91098721 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.202361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 470/933] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:33.203946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66097a07-e369-4933-914b-218c91098721 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.240422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.259206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d74c5-28b0-4768-b6b4-5a42800ae536 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.277565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.299086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.299936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.300215 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.300523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.301164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.302555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:33.315181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:33.316206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.317052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 464/934] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:33.317867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.325298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66097a07-e369-4933-914b-218c91098721 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.362901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.363583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.363880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.364184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.364897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:33.365491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.366694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.371821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 471/935] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:33.377123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.427576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081beda-d4d5-4cd9-86ba-5c60fa453956 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.464616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.465422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.465703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.465978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.473082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.474546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.482097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.486646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa50178-e311-46d1-a720-1d61b70653b5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:33.492784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:33.493756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:33.494645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:33.499262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 465/936] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:33.500103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.525507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.526448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.526890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.527168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.527904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:33.528207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.529608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.534780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 472/937] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:33.535563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.551859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.620021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c047a0-2f20-412d-bb7b-471cab781227 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.643201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e594-58f4-429e-b040-e01ae82958b1 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.656519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.657173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.657453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.657749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.658445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:33.658752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.660158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.665418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 466/938] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:33.666201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.682988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.683701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.683935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.684212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.684999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:33.685308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.686733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.699744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 473/939] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:33.699744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.729338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.780065 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce76ac-e38d-469a-9de5-c605f67c55c1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.788052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:33.811570 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba681a5-2892-4505-b321-48e53b40305e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.817703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.818377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.818728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.819008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.819757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:33.820100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.821487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.826542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 467/940] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:33.827312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.861268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:33.862383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:33.862663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:33.862936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:33.863677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:33.865071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:33.895995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 474/941] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:33.895995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:33.966769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d419a-88fe-414e-adf4-5975dcdfcc72 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:33.983080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:34.015119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:34.015953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2a589-c9a5-407e-820b-888f31dadbe9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:34.016777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.017277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.017880 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/cc7643dc9f834584973cb7fdedbb4d70/roles/f39bdab856724510a905efb68870ac32` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.020669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=cc7643dc9f834584973cb7fdedbb4d70, role_id=f39bdab856724510a905efb68870ac32)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:34.022826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:34.048207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 468/942] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:33 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/cc7643dc9f834584973cb7fdedbb4d70/roles/f39bdab856724510a905efb68870ac32 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:48:34.059354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:34.060070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.060345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.060619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.061427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] RBAC: Adding query filter params (interface=public, service_id=8e869dba63c5478db54a63f37984797e, region_id=RegionOne) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:34.061741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:34.063231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3afc0-99b1-4753-9923-d4161eb6b192 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:34.069449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 475/943] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:33 2022] GET /identity/v3/endpoints?service_id=8e869dba63c5478db54a63f37984797e&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:34.076505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:34.114434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:34.202787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67262aab-3fc3-4c33-9ca9-51e8080c9a9b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:34.241378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcec7a3-8a54-46d3-9fa9-204edfc9aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:34.242408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcec7a3-8a54-46d3-9fa9-204edfc9aa6b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.242918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcec7a3-8a54-46d3-9fa9-204edfc9aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.243450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcec7a3-8a54-46d3-9fa9-204edfc9aa6b None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.244446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcec7a3-8a54-46d3-9fa9-204edfc9aa6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:34.246083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcec7a3-8a54-46d3-9fa9-204edfc9aa6b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:34.252978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 469/944] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:34 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:48:34.411234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c65cd3-0f9e-48b7-a771-0f92087e5feb None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.411631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c65cd3-0f9e-48b7-a771-0f92087e5feb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.411791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c65cd3-0f9e-48b7-a771-0f92087e5feb None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.413507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 476/945] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:34.418778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.419070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.419350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.437903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:34.439154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:34.508841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 470/946] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:34.532386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c65cd3-0f9e-48b7-a771-0f92087e5feb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:34.637553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c65cd3-0f9e-48b7-a771-0f92087e5feb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:34.676792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:34.677455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.677733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.678074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.678793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:34.680300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:34.681676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING keystone.server.flask.common [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 09:48:34.698960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:34.704350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.sql.core [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 09:48:34.704350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 09:48:34.704350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [parameters: {'id': 'fcbc7c84a2c34b4fa191699b724b3027', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 09:48:34.704350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80149) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fcbc7c84a2c34b4fa191699b724b3027.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fcbc7c84a2c34b4fa191699b724b3027. Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application conn.query(q) Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 09:48:34.723459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application result.read() Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 09:48:34.724545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application next(self.gen) Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application yield resource Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application next(self.gen) Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise self.value Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application session.commit() Jun 30 09:48:34.725860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self.session.flush() Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application compat.raise_( Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 09:48:34.726939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 09:48:34.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application conn.query(q) Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application result.read() Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 09:48:34.728893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [parameters: {'id': 'fcbc7c84a2c34b4fa191699b724b3027', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:34.729837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:34.730851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fcbc7c84a2c34b4fa191699b724b3027. Jun 30 09:48:34.732076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:34.734747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 477/947] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Jun 30 09:48:34 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:34.740500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:34.764674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:34.844068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea1397-f402-4643-a280-58d7469e1db1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:34.878127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:34.878926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:34.879204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:34.879480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:34.886318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:34.887779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-16ea5978-1fe8-470e-b8d7-fe1e4f3926b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:34.901605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:34.903636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:34.905741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 471/948] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:34.913265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:35.019138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79219d17-1b87-41c6-92ed-d9d7a7942666 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:35.054920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:35.055588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:35.055880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:35.056188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:35.056952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:35.057257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:35.058908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdb423-1f37-4a47-887d-5693c882a126 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:35.079272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 478/949] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:36.071280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b3c30-bf9c-4a9b-889c-f3f228ff1b14 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.071807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b3c30-bf9c-4a9b-889c-f3f228ff1b14 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.071807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b3c30-bf9c-4a9b-889c-f3f228ff1b14 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.073770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 472/950] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:36.079260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ceef4-d148-401c-8d4c-0a9e9ad0e2f8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.079688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ceef4-d148-401c-8d4c-0a9e9ad0e2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.080045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ceef4-d148-401c-8d4c-0a9e9ad0e2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.097850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bb8ceef4-d148-401c-8d4c-0a9e9ad0e2f8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:36.099355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bb8ceef4-d148-401c-8d4c-0a9e9ad0e2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:36.149030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:36.168017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 479/951] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:36.190378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b3c30-bf9c-4a9b-889c-f3f228ff1b14 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.190935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357e318-02ed-4fed-b710-7571813d7ba9 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.191228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357e318-02ed-4fed-b710-7571813d7ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.191510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357e318-02ed-4fed-b710-7571813d7ba9 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.193178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 480/952] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:36.198944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.199217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.199494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.220587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:36.221783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:36.287677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:36.296007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 481/953] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:36.312259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b3c30-bf9c-4a9b-889c-f3f228ff1b14 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.318816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.348733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.349535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.349879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.350356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.351093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:36.352517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-4c9c484e-451f-44c0-b632-0af9c9cf248e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:36.364904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:36.365995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:36.367015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:36.371437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 473/954] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:36.372713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1970a-687f-4a42-85fb-07958d346c07 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.373001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1970a-687f-4a42-85fb-07958d346c07 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.373285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1970a-687f-4a42-85fb-07958d346c07 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.374578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 474/955] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:36.375270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1970a-687f-4a42-85fb-07958d346c07 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.429074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dcbc9-1034-400f-817d-acd235b80778 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.444119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:36.466805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715cb5e-d3b2-4d48-87b9-8fd10a55c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.467773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715cb5e-d3b2-4d48-87b9-8fd10a55c480 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.468052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715cb5e-d3b2-4d48-87b9-8fd10a55c480 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.468324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715cb5e-d3b2-4d48-87b9-8fd10a55c480 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.479263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715cb5e-d3b2-4d48-87b9-8fd10a55c480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:36.480714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1970a-687f-4a42-85fb-07958d346c07 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-9715cb5e-d3b2-4d48-87b9-8fd10a55c480 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:36.487619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:36.488665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:36.489494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:36.491944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 482/956] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:36.493092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.493377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.493785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.510875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:36.512183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:36.523905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.524598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.524887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.525175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.525889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:36.526194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:36.527571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:36.532462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 475/957] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:36.533237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.548125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:36.592203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 483/958] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:36.593888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.646366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab2f55-2dbf-4e83-96ec-8b72102ab945 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.682616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.683300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.683583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.683859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.684650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:36.685306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:36.686457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:36.701113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 476/959] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:36.701113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.726062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef271e96-acb9-4621-9368-e2ba99d5d284 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.728773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:36.765232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.765905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.766183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.766455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.767172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:36.767474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:36.768887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:36.773633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 484/960] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:36.776555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:36.780850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.820567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2f28d-075a-41c7-b7af-b5c7215325d6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.873862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.875129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.875618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.876138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.877708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:36.879820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:36.897900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:36.898982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:36.899830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:36.906533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 477/961] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:36.907518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:36.908924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d328a-5c35-4f6e-9617-a668785b835f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:36.947585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:36.949195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:36.949481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:36.949991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:36.962351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:36.963818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:36.984645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:36.985617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:36.986503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:36.986503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:36.986503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:36.986503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:36.986503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 485/962] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:36.991489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.013727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.029693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090db6ae-3f99-4d1e-a91d-1175991a2677 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.066991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.067664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.068014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.068433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.069164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:37.069465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.070966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.072171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.078144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 478/963] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:37.084338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.102874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ce8df-0d02-4d68-b577-37426576e340 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.140873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.141904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.142347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.142809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.143676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:37.144194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.145836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.162111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 486/964] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:37.167857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.204492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb22b9-00b8-4ece-8a53-ef8b7e8d4b82 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.240349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.241333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.241796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.242245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.243159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:37.243656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.245242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d142d5-73f7-4c59-aa5f-d6c7ae783d9a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.250278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 479/965] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:37.251780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305eebc-4282-4729-934d-81915e13d531 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.252269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305eebc-4282-4729-934d-81915e13d531 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.252733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305eebc-4282-4729-934d-81915e13d531 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.254422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 480/966] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:37.255602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305eebc-4282-4729-934d-81915e13d531 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.266391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.289007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832241c-68f2-4d7d-8ffd-245b1489f1d9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.325223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.326136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.326495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.326774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.330395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.335042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.336877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-5334b25a-06a9-4623-af3a-985dfb0d348f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:37.351637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:37.352697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:37.353655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 487/967] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:48:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:37.354949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.354949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.354949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.372404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:37.373585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:37.379382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305eebc-4282-4729-934d-81915e13d531 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.417329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.417903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.418183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.418455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.419299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.420811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.447933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 488/968] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:37.447933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.449897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 481/969] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:37.470375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.557345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.600101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af05f2-f396-4c68-8b47-44712ac49ccc None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.604238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e213a9-c738-45fe-8d8b-ec8aab5450f5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.619558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.642700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.643031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.643533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.643846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.644025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.644260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.644376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.644637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.645195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.645601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:37.645953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.647042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.647627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.648525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING keystone.server.flask.common [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 09:48:37.665042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.sql.core [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 09:48:37.665042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 09:48:37.665042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [parameters: {'user_id': 'ee5a7f5c2b7a41f3a307fd0c7c8b7b08', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 09:48:37.665042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80150) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 09:48:37.668223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 489/970] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 09:48:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Jun 30 09:48:37.673175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 09:48:37.674244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application yield resource Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application next(self.gen) Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise self.value Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application session.commit() Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 09:48:37.675397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.session.flush() Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application compat.raise_( Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 09:48:37.676982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 09:48:37.678998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application conn.query(q) Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application result.read() Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 09:48:37.681819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ee5a7f5c2b7a41f3a307fd0c7c8b7b08', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:37.683689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 09:48:37.688746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:37.690809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 482/971] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 09:48:37 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:37.692158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.694460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.804699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653b7aa-d34e-4315-9276-870c1a4ada88 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.824158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eafa42-ddfa-49a0-8230-a61382cd41e0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:37.842490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.843685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.844178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.844638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.853031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.855020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:37.857134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.867000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:37.867730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:37.868099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:37.868433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:37.873643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:37.874742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 09:48:37.875930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:37.882989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 483/972] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:37.884597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:37.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:37.958516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:37.961731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.006704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63704ced-21be-4b84-9f16-84dae8359cc7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:38.046618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:38.047448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.047802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.048167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.049056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:38.049456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:38.051920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.091425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 490/973] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:38.098309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:38.098630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 484/974] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 09:48:37 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:38.128144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:38.215228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7a92e-a9cc-49ab-8ce3-5a1126b3e4eb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:38.253052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:38.253858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.254133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.254404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.265060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:38.266519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:38.283919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:38.284838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:38.286003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:38.286003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:38.286003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:38.286003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:38.286003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 491/975] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:38.291356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:38.395331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:38.402686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b110e-20c2-4be2-b1c5-59c4c0f7d8a8 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:38.437320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:38.437991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.438271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.438543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.439272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:38.439575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:38.440993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.453546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 485/976] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:38.459039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:38.544601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:38.565391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f647-511f-4030-83e3-898c4d935a4b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:38.602414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:38.603238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.603517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.603789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.611270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:38.612758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:38.627313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:38.628437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:38.629365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:38.631956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 492/977] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:38.634744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:38.736364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69f94a-bfcc-4616-b123-3a77baa158bd None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:38.770108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:38.770777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.771061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.771446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.772317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:38.772627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:38.774072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c327c-fc39-4078-9d29-3779f754ab95 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.786418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 486/978] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:38.792265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:38.891726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737ad44-fdb8-4bcc-9c7d-7bcfac2cde19 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:38.906180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082a78-5fc1-4ea0-869d-072c5cbae53b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.906468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082a78-5fc1-4ea0-869d-072c5cbae53b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.906736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082a78-5fc1-4ea0-869d-072c5cbae53b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.908420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 487/979] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:38.912943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed0c5-501e-4fec-ab4d-d5a2dce538fa None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.913227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed0c5-501e-4fec-ab4d-d5a2dce538fa None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.913504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed0c5-501e-4fec-ab4d-d5a2dce538fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.930103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5fbed0c5-501e-4fec-ab4d-d5a2dce538fa None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:38.931697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5fbed0c5-501e-4fec-ab4d-d5a2dce538fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:38.932571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:38.933230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:38.933494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:38.933755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:38.934601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] RBAC: Adding query filter params (role.id=f39bdab856724510a905efb68870ac32, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=cc7643dc9f834584973cb7fdedbb4d70) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:38.934902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:38.936390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:38.941864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:38.942804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 493/980] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:38 2022] GET /identity/v3/role_assignments?user.id=cc7643dc9f834584973cb7fdedbb4d70&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=f39bdab856724510a905efb68870ac32 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:39.018213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 488/981] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:39.041381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:39.045526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:39.154366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0a880-36fb-4603-a29f-736e78604d9d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:39.188906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706672d9-0b3b-4141-8a57-6037e52fab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:39.189599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706672d9-0b3b-4141-8a57-6037e52fab19 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:39.189855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706672d9-0b3b-4141-8a57-6037e52fab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:39.190126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706672d9-0b3b-4141-8a57-6037e52fab19 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:39.190858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706672d9-0b3b-4141-8a57-6037e52fab19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:39.192259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706672d9-0b3b-4141-8a57-6037e52fab19 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:39.215910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 494/982] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:39 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:39.519390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd878a-0682-450f-be16-0f5251308127 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:39.519939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd878a-0682-450f-be16-0f5251308127 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:39.519939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd878a-0682-450f-be16-0f5251308127 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:39.521732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 489/983] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:39.526786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:39.527037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:39.527350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:39.544569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:39.545725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:39.608756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:39.626276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 495/984] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:39.650141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd878a-0682-450f-be16-0f5251308127 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:39.758733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd878a-0682-450f-be16-0f5251308127 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:39.792735 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:39.793462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:39.793744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:39.794072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:39.794728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:39.796199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:39.808232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:39.809235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:39.810087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:39.812041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 490/985] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:39.815388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:39.914727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f813748e-9cc4-4ad1-8c35-2c44783ae1fd None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:39.950041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:39.950868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:39.951151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:39.951422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:39.952179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:39.952485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:39.953866 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:39.959944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 496/986] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:39.965053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:39.976888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:40.068446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6212185-6079-4ace-84bb-659a2bf625b4 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:40.104092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:40.104766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:40.105045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:40.105419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:40.106031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:40.106334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:40.107712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:40.116251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 491/987] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:40.118700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:40.176985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:40.224417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0849b9c-6505-4c9a-8619-74ff3ea82133 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:40.259279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:40.259966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:40.260250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:40.260520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:40.261325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] RBAC: Adding query filter params (interface=admin, service_id=8e869dba63c5478db54a63f37984797e, region_id=RegionOne) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:40.261642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:40.263019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d95b29-a555-4d3e-a0b2-f4038f56eb23 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:40.268841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 497/988] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Jun 30 09:48:40 2022] GET /identity/v3/endpoints?service_id=8e869dba63c5478db54a63f37984797e&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:40.275197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:40.355526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:40.387263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9541b-fe7a-4fc9-a4aa-bdb0eff27e1c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:40.425619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422b1ff-81e2-4b2e-acb4-56a47f007b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:40.426301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b1ff-81e2-4b2e-acb4-56a47f007b1b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:40.426586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b1ff-81e2-4b2e-acb4-56a47f007b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:40.426863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422b1ff-81e2-4b2e-acb4-56a47f007b1b None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:40.427553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422b1ff-81e2-4b2e-acb4-56a47f007b1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:40.429045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422b1ff-81e2-4b2e-acb4-56a47f007b1b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:40.439935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 492/989] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:40 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:48:41.007563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cc12c-cdf2-4301-a82b-a030b158d745 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.008019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cc12c-cdf2-4301-a82b-a030b158d745 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.008155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cc12c-cdf2-4301-a82b-a030b158d745 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.009763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 498/990] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:41.015194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cc023-6df2-445b-b27f-1474bf1a6712 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.015668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cc023-6df2-445b-b27f-1474bf1a6712 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.015746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cc023-6df2-445b-b27f-1474bf1a6712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.033509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-de5cc023-6df2-445b-b27f-1474bf1a6712 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:41.034711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-de5cc023-6df2-445b-b27f-1474bf1a6712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:41.108794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 493/991] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:41.131127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535cc12c-cdf2-4301-a82b-a030b158d745 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.220353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2091045-29f5-4f87-b1ee-b3562f6f3700 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.220642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2091045-29f5-4f87-b1ee-b3562f6f3700 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.220919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2091045-29f5-4f87-b1ee-b3562f6f3700 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.222324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 494/992] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:41.227495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.227754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.228105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.242837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535cc12c-cdf2-4301-a82b-a030b158d745 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.245907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:41.247092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:41.263358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:41.286713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.290426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.290426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.290426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] PATH_INFO: `/v3/services/image` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.290426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.290979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:41.305447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:41.306480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:41.307419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 499/993] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:41.312447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.314041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:41.323912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 495/994] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:41.345824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.424665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab440b76-b721-4c15-a279-575ad1992347 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.454189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7ff38-bb3b-44f6-8b24-040b6aee8040 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.459588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.460297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.460924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.461201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.461916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] RBAC: Adding query filter params (name=image) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:41.462219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.463601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:41.469381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 500/995] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:41.474857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.492350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.493184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.493518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.493699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.494423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.495741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:41.508113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:41.509116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:41.509962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:41.514250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 496/996] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:41.515042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.528951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:41.580244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:41.587372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0200e-e660-45ac-a8b0-15cd2e6fab01 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.624936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.625608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.625919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.626186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.626855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:41.627163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.628007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002adc-1040-4ee3-94b5-afab133b9f06 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.628572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:41.636201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 501/997] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:41.665777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.666472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.666758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.667036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.667771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:41.668135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.669564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:41.675384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 497/998] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:41.683569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.798522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5969b-84bc-4974-9a32-1450a84ca9a5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.809216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:41.833936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.834684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.834971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.835844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.836384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:41.836694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.838074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:41.847755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 502/999] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:41.849205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:41.955533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ebd79-57f1-4586-9be9-d53fdc17af62 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:41.980563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:41.992266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:41.992942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:41.993222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:41.993500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:41.994301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] RBAC: Adding query filter params (interface=public, service_id=1cac07683324426cbd671fadf6e7ce3d, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:41.994616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:41.996025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c93d2-8458-428a-984f-7e1209dde02f None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:42.001749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 498/1000] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:41 2022] GET /identity/v3/endpoints?service_id=1cac07683324426cbd671fadf6e7ce3d&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:42.007647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:42.115856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9ffbb-744b-469c-839c-f65ff40b96cd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:42.153493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15214e8a-0def-421e-b2e7-d1b1d831cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:42.154172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15214e8a-0def-421e-b2e7-d1b1d831cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:42.154455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15214e8a-0def-421e-b2e7-d1b1d831cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:42.154732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15214e8a-0def-421e-b2e7-d1b1d831cb3e None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:42.155420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15214e8a-0def-421e-b2e7-d1b1d831cb3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:42.156843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15214e8a-0def-421e-b2e7-d1b1d831cb3e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:42.163369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 503/1001] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:42 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 09:48:42.449807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b55ea-239d-4ca6-803d-5ec6eeb18091 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:42.450356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b55ea-239d-4ca6-803d-5ec6eeb18091 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:42.450356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b55ea-239d-4ca6-803d-5ec6eeb18091 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:42.451621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 499/1002] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:42.457037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:42.457441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:42.457580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:42.476121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:42.477295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:42.507791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:42.546247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 504/1003] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:42.569427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b55ea-239d-4ca6-803d-5ec6eeb18091 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:42.675012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b55ea-239d-4ca6-803d-5ec6eeb18091 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:42.711290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:42.712062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:42.712347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:42.712615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:42.713273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:42.714642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:42.726809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:42.727805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:42.729435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 500/1004] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 09:48:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:42.737264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:42.839462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52832e-a24f-42d0-86e2-02b95e437881 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:42.874092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:42.874771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:42.875023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:42.875314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:42.876035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:42.876337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:42.877713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:42.883250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 505/1005] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:42.888831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:42.941181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:42.989316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046007-6938-4659-87ec-703f8db6b7ed None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:43.022990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:43.023644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:43.023933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:43.024218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:43.024921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:43.025223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:43.026588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ea6b2-23f0-44de-81c1-4d4e8b0cf031 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:43.031619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 501/1006] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 09:48:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:43.037455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:43.067982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:43.138550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525483f-d629-44ee-8cbc-5947bc9ec41a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:43.171795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cb4e5-b749-49ab-999e-91745fbcf399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:43.172483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb4e5-b749-49ab-999e-91745fbcf399 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:43.172763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb4e5-b749-49ab-999e-91745fbcf399 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:43.173030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cb4e5-b749-49ab-999e-91745fbcf399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:43.173772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cb4e5-b749-49ab-999e-91745fbcf399 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:43.175135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cb4e5-b749-49ab-999e-91745fbcf399 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:43.194045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 506/1007] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:48:43.661769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093dca8-ccc6-4620-bcd8-582a6ae8efef None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:43.662526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093dca8-ccc6-4620-bcd8-582a6ae8efef None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:43.662526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093dca8-ccc6-4620-bcd8-582a6ae8efef None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:43.663728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 502/1008] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:43.669310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac198d9f-c1a2-4934-96c5-4d1cfc710d1f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:43.669737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac198d9f-c1a2-4934-96c5-4d1cfc710d1f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:43.669861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac198d9f-c1a2-4934-96c5-4d1cfc710d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:43.686941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ac198d9f-c1a2-4934-96c5-4d1cfc710d1f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:43.688142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ac198d9f-c1a2-4934-96c5-4d1cfc710d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:43.763233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 507/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:43 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:43.785904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093dca8-ccc6-4620-bcd8-582a6ae8efef None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:43.891227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093dca8-ccc6-4620-bcd8-582a6ae8efef None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:43.908316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:43.927665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f578e-7e49-4dc0-8d8a-7cf66827546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:43.928281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f578e-7e49-4dc0-8d8a-7cf66827546b None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:43.928557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f578e-7e49-4dc0-8d8a-7cf66827546b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:43.928832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f578e-7e49-4dc0-8d8a-7cf66827546b None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:43.929572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f578e-7e49-4dc0-8d8a-7cf66827546b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:43.931018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f578e-7e49-4dc0-8d8a-7cf66827546b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:43.955901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 503/1010] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:43 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:44.190643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc497f24-2559-4c76-9389-d35d97e5bf5c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:44.190822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc497f24-2559-4c76-9389-d35d97e5bf5c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:44.191095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc497f24-2559-4c76-9389-d35d97e5bf5c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:44.192666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 508/1011] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:44.197933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:44.198192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:44.198498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:44.216141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:44.217312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:44.301836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 504/1012] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:44 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:44.325029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc497f24-2559-4c76-9389-d35d97e5bf5c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:44.345213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:44.431024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc497f24-2559-4c76-9389-d35d97e5bf5c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:44.464916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:44.465680 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:44.465958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:44.466241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:44.466929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:44.468336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:44.480220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:44.481187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:44.487187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 509/1013] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:44.489581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:44.590273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32991cb-4162-404e-b0a6-5d5713e78fd5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:44.591777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:44.629190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:44.629858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:44.630138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:44.630403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:44.631179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:44.631482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:44.632890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:44.638178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 505/1014] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:44.643749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:44.728546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:44.748406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb2d54-3ca2-452b-9669-6defb8af12c1 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:44.781169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:44.781831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:44.782110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:44.782379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:44.783083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:44.783388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:44.784784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0910c-f5ac-4517-a2ae-334fc9f8faeb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:44.789804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 510/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 09:48:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:44.795378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:44.892866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a87ed-3636-4049-8f0e-396bce4d1481 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:44.926112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2fcd1-1cf0-4bb8-9384-1046a8711abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:44.926766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2fcd1-1cf0-4bb8-9384-1046a8711abe None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:44.927047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2fcd1-1cf0-4bb8-9384-1046a8711abe None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:44.927324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2fcd1-1cf0-4bb8-9384-1046a8711abe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:44.928197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2fcd1-1cf0-4bb8-9384-1046a8711abe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:44.929564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2fcd1-1cf0-4bb8-9384-1046a8711abe None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:44.951964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 506/1016] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:45.609763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28736-eb19-4327-8188-6806b1bb861f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:45.610265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28736-eb19-4327-8188-6806b1bb861f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:45.610757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28736-eb19-4327-8188-6806b1bb861f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:45.613488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 511/1017] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:45.621984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298247f2-54da-417a-a395-786b5aa97020 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:45.622502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298247f2-54da-417a-a395-786b5aa97020 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:45.622992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298247f2-54da-417a-a395-786b5aa97020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:45.650398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-298247f2-54da-417a-a395-786b5aa97020 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:45.652455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-298247f2-54da-417a-a395-786b5aa97020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:45.681098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:45.780092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 507/1018] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:45.817924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c28736-eb19-4327-8188-6806b1bb861f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:45.993101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c28736-eb19-4327-8188-6806b1bb861f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:46.049935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b48f5-92dd-4736-9a24-73d50c2e4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:46.050822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b48f5-92dd-4736-9a24-73d50c2e4722 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:46.051103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b48f5-92dd-4736-9a24-73d50c2e4722 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:46.051370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b48f5-92dd-4736-9a24-73d50c2e4722 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:46.052759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b48f5-92dd-4736-9a24-73d50c2e4722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:46.054210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b48f5-92dd-4736-9a24-73d50c2e4722 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:46.059898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 512/1019] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:46.061931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298247f2-54da-417a-a395-786b5aa97020 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:46.167042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298247f2-54da-417a-a395-786b5aa97020 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:46.202151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ac88d-ba55-4c42-a217-46355c890b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:46.202816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac88d-ba55-4c42-a217-46355c890b07 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:46.203095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac88d-ba55-4c42-a217-46355c890b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:46.203369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ac88d-ba55-4c42-a217-46355c890b07 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:46.204126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ac88d-ba55-4c42-a217-46355c890b07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:46.205579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ac88d-ba55-4c42-a217-46355c890b07 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:46.227232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:46.235279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 508/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:48:46 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:46.560428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9fd2-17e2-4915-84bf-b5577c0a8ff7 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:46.560638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9fd2-17e2-4915-84bf-b5577c0a8ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:46.560890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f9fd2-17e2-4915-84bf-b5577c0a8ff7 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:46.562288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 513/1021] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:46.567841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:46.568202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:46.568479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:46.586262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:46.587453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:46.659115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 509/1022] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:46.683551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f9fd2-17e2-4915-84bf-b5577c0a8ff7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:46.700701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:46.788909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f9fd2-17e2-4915-84bf-b5577c0a8ff7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:46.822222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:46.822958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:46.823211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:46.823477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:46.824167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:46.825542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:46.837715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:46.838691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:46.839531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:46.841075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 514/1023] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jun 30 09:48:46 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:46.845023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:46.949857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6faf45-8c96-45d5-9ea5-5c3a8578ca76 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:46.979891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:46.987300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:46.988003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:46.988289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:46.988566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:46.989275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] RBAC: Adding query filter params (name=image) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:46.989581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:46.991018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c982-b7a5-4f3c-bf1a-049c58413f89 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:46.996396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 510/1024] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:47.001995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.063365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a1d03-a5c8-4dd0-83d1-dccb0233a759 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.063975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a1d03-a5c8-4dd0-83d1-dccb0233a759 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.064215 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a1d03-a5c8-4dd0-83d1-dccb0233a759 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.065604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 511/1025] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:47.070543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.070815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.071112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.083412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:47.088824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:47.090018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:47.109091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f30a5-3d13-4ee4-b5bc-3a122d3631bd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.146141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.146894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.147175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.147449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.148190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] RBAC: Adding query filter params (type=image) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:47.148498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:47.149888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:47.155137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 515/1026] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:47.160956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.169449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 512/1027] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:47.195682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.266605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83767-9ec2-42d6-bc0d-55d7c3d4043b None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.303531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.304304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.304585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.304860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.305658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] RBAC: Adding query filter params (interface=public, service_id=1afc4be4f0c141eebf7ed028f83f2d03, region_id=RegionOne) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:47.305972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:47.307354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:47.312644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 516/1028] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/endpoints?service_id=1afc4be4f0c141eebf7ed028f83f2d03&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:47.319005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.320740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f188af4-658f-46d6-8bdb-459b6e8eb6b3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.329184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:47.345397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:47.357258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.358119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.358385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.358658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.370548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:47.378730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:47.379934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:47.381273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:47.385336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 513/1029] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:47.386167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.445246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b1c61-4fb0-41d7-be2a-7d81f85caf6e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.483059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.483728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.484054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.484313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.485176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:47.486376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:47.487987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a884b-d207-4fcf-a47e-9460a3aa31a1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.493143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 517/1030] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:48:47.540166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.540853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.541136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.541412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.542185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:47.542493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:47.543994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:47.553996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 514/1031] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:47.559618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.634544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:47.670434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314babc2-8c09-4976-aea3-4e8b709c8f39 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.703473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.704292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.704569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.704836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.716053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:47.717502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:47.734181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:47.735138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:47.736454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:47.736454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:47.736454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:47.736454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:47.736454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 518/1032] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:47.741717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:47.792833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:47.850221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582dc-671e-4fd9-a590-5f1e30a1f245 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:47.884246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:47.884970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:47.885192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:47.885464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:47.886193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:47.886496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:47.887891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:47.900102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 515/1033] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:47.905644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.007545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136572d-2382-4aee-9897-5f0722af9617 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.029882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:48.042629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.043437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.043716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.044015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.051781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.053330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:48.067213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:48.068447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:48.069391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:48.072248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 519/1034] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:48.074525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.179108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56675c-5fe7-408d-a737-160697745d9a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.186585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:48.215426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.216123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.216401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.216671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.217506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:48.217810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.219273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8702c-27d3-40cb-bcb2-07e5b60a3366 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.236751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 516/1035] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:48.242391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.289147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcef86-680a-4ebe-b4c3-e0bf275eaa62 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.289484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcef86-680a-4ebe-b4c3-e0bf275eaa62 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.289702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcef86-680a-4ebe-b4c3-e0bf275eaa62 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.292647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 517/1036] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:48:48.296367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.296654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.296929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.314046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:48.315806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:48.353841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f52b3f-fe2c-4890-9d11-d9c45287f24e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.385923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 518/1037] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:48.388771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.389425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.389723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.389978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.410660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.440790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:48.461599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.463060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.520317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:48.528467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0960790-2758-4b6f-9ca1-bf5e5ef99741 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.568384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 520/1038] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:48.568580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.568580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.568580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.568580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.568877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.568959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.569507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.572088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 519/1039] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:48.576150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.656772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:48.675330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5ab24-2cac-42e7-801f-48be27eca322 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.695679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11574598-1732-445d-92d6-c9ddb444e132 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.710311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.711459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.711738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.712037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.723362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.724896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.732372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.733273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.733556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.734136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.734555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.736010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:48.743047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:48.743926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:48.744831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:48.744831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:48.744831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:48.744831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:48.744831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 521/1040] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:48.749839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.767907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 520/1041] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:48:48 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:48:48.859122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b082f-b785-41a3-92dd-f0980b0b0451 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:48.896768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:48.897476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:48.897753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:48.898021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:48.898759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:48.899067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:48.900499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:48.902266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:48.914744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 522/1042] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:48.920302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:48.941314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:49.026603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7b77b-3d55-4383-bb17-c5f77eaf8b16 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:49.061462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:49.062259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.062536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.062801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.069846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:49.071302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:49.086256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:49.087630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:49.088574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 521/1043] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:49.093567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:49.196047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b8705-d84a-41da-9a23-42b4755a4c3d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:49.230334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:49.230991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.231276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.231543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.232357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:49.232673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:49.234110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b8a8-6a8e-4748-b8d2-53a345afcfc0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:49.246814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 523/1044] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:49.252743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:49.340510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:49.363236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e291aa-bd24-40cb-afa3-003bb57b16e6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:49.397535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:49.398186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.398464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.398735 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.399566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] RBAC: Adding query filter params (role.id=f39bdab856724510a905efb68870ac32, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=14dd1de1fe914f319017818e571732de) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:49.399892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:49.401328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feabdc2-4877-4be7-a887-454bea317f12 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:49.407166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 522/1045] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:49 2022] GET /identity/v3/role_assignments?user.id=14dd1de1fe914f319017818e571732de&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=f39bdab856724510a905efb68870ac32 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:49.561402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3172-63d2-4dce-b609-4605dfa09f7e None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.561597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3172-63d2-4dce-b609-4605dfa09f7e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.561826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3172-63d2-4dce-b609-4605dfa09f7e None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.563355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 524/1046] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:49.568956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.569242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.569519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.586851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:49.588071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:49.658063 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 523/1047] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:49.681654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3172-63d2-4dce-b609-4605dfa09f7e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:49.739796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:49.785958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3172-63d2-4dce-b609-4605dfa09f7e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:49.819614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:49.820351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.820622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.820890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] PATH_INFO: `/v3/services/image` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.821543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:49.823040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:49.834742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:49.836148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:49.837021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 525/1048] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jun 30 09:48:49 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:49.844085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:49.918808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:49.952723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5c1c5-4e8f-40de-a3a5-392e10695747 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:49.989502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:49.990218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:49.990487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:49.990800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:49.991524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:49.991823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:49.993247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:50.004091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 524/1049] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:50.004353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:50.110235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a21cc-8551-4388-aa8b-794e0f48223d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:50.128076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:50.147668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:50.148370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:50.148652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:50.148926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:50.149639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] RBAC: Adding query filter params (type=image) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:50.149946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:50.151437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff12b0c-4e6b-4e2a-a77e-53a53a74aa38 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:50.156637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 526/1050] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:50.162377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:50.265256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05928654-2eef-4107-a331-c12bf754f0a6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:50.304397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8274179-67d1-43e3-b4ad-19c33462f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:50.305066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8274179-67d1-43e3-b4ad-19c33462f00d None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:50.305345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8274179-67d1-43e3-b4ad-19c33462f00d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:50.305618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8274179-67d1-43e3-b4ad-19c33462f00d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:50.306364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8274179-67d1-43e3-b4ad-19c33462f00d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:50.307750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8274179-67d1-43e3-b4ad-19c33462f00d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:50.312114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:50.328312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 525/1051] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:48:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:48:50.864156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d1c76-d423-48b6-952b-3453613e80ed None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:50.864571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d1c76-d423-48b6-952b-3453613e80ed None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:50.864784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d1c76-d423-48b6-952b-3453613e80ed None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:50.866145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 527/1052] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:50.871889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:50.872170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:50.872480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:50.889846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:50.891030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:50.966284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 526/1053] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:50.990071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d1c76-d423-48b6-952b-3453613e80ed None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.098755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d1c76-d423-48b6-952b-3453613e80ed None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.133979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.134699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.134981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.135254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.146905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:51.154422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:51.155368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:51.156243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:51.160221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 528/1054] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:51.161698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.263811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e153ffe-6bcd-414d-85c9-0a53efbcfe82 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.284597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:51.306883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.307567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.307853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.308156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.308920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:51.309230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:51.310618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555f3a-820b-4908-a34a-7b467ac30398 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:51.319523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 527/1055] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:51.325161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.370927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:51.388189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1baa0-240c-421c-88b1-acce557f91b7 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.388729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1baa0-240c-421c-88b1-acce557f91b7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.389010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1baa0-240c-421c-88b1-acce557f91b7 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.390466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 528/1056] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:51.395351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.395968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.396476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.415072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:51.416860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:51.430607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6280b-6a0f-4290-b06d-0c3d1393ff05 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.470401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.471279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.471556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.471824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] PATH_INFO: `/v3/users/5e61f06f1d684d1aba97c2d85c09839e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.485800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] RBAC: Authorizing `identity:get_user(user_id=5e61f06f1d684d1aba97c2d85c09839e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:51.487741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:51.490099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 529/1057] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/users/5e61f06f1d684d1aba97c2d85c09839e => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:51.496023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.510647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 529/1058] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:51.532869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.594628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:51.597527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae946f6-2c11-449a-a9ee-866fa332beb1 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.639995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.641066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.641400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.641819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.651032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784dee9-ffd1-4e9d-8818-4e663710c0f7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.662329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:51.688687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.689759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.690214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.690780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.701740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:51.708999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:51.710829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:51.711379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:51.712528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:51.713388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 530/1059] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:51.718731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.811219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:51.816300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 530/1060] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 09:48:51.821928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.828449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c21c0-fd21-4c64-8ac1-edff1a3d68f0 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.866025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.866702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.866980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.867251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.868042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:51.868350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:51.869715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:51.877952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 531/1061] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:51.884052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.916808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:51.927579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484911-4ef8-4af0-981b-bee2c8c89130 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:51.963342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:51.964390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:51.964670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:51.964938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:51.966732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] RBAC: Authorizing `identity:get_project(project_id=dbb51914ad5845c3a446c74365f1fd33)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:51.967580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:51.969480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 531/1062] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:51.975825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:51.999507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873342e-e836-4126-9fe7-e1d1e70eacd3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.038204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.039400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.039752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.040064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.044719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.051911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.053581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:52.071391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:52.072439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:52.073361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:52.073361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:52.073361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:52.073361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:52.073361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 532/1063] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:52.078654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.083338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460cdfe-5f2b-4ccd-a781-55614be20096 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.120425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.121078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.121326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.121602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.122428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=dbb51914ad5845c3a446c74365f1fd33, user.id=5e61f06f1d684d1aba97c2d85c09839e) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:52.122787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.124268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.130108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 532/1064] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:51 2022] GET /identity/v3/role_assignments?user.id=5e61f06f1d684d1aba97c2d85c09839e&scope.project.id=dbb51914ad5845c3a446c74365f1fd33&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:52.173983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.193431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e690-7cc1-4526-94df-fefd3d00fe38 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.233028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.233720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.234005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.234280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.235021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:52.235328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.236759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ac96-5db1-4a85-bcaa-49c5da1024ff None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.255845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 533/1065] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:52.257065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.358053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d70b6-0262-49a5-be6a-3902de71ce99 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.358862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d70b6-0262-49a5-be6a-3902de71ce99 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.359142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d70b6-0262-49a5-be6a-3902de71ce99 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.361291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 534/1066] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:52.366608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.366886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.367388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.370715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3214b6-4a42-485d-9226-447bbfabf1b7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.388291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:52.389849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:52.401538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.409599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.410542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.410790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.411086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.418750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.420236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:52.434799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:52.435753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:52.437076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 533/1067] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:52.441621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.479912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 535/1068] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:52.498447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.545867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949668df-3a5e-49c2-bfab-d0fedbd766cb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.575224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.583997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.584678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.584954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.585223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.586408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:52.586711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.588157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.605560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030f181-9249-44d9-ad3c-cd91aa00d648 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.610309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 534/1069] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:52.611246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.646447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.647218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.647562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.647938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.662045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.713967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a362-0af1-48e9-abed-924c6d21dee6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.720225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.721795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.752299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.753101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.753374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.753639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.765153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.766612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:52.785262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:52.786192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:52.787312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:52.787312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:52.787312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:52.787312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:52.787312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 535/1070] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:52.792341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.805406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.836038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 536/1071] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:52.838008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.898258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82d5d-c2f6-438e-bc35-f11ef3d1ca15 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.899899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:52.934303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.934966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.935246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.935509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.936255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:52.936566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.937935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:52.943271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d061ee-ea40-4ed4-b4a3-984f9f6afd6d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:52.949754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 536/1072] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:52.955108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:52.979029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:52.979750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:52.980062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:52.980471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:52.991202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:52.992684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:53.012169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:53.013077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:53.013973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:53.013973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:53.013973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:53.013973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:53.013973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 537/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:53.019240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:53.051753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:53.068376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f0e6-4548-4f44-a393-24309adfd12f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:53.105231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:53.106043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:53.106325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:53.106601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:53.113561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:53.115285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:53.129457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:53.130980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:53.131927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 537/1074] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:53.136680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142307c-319d-4985-a531-b169c7e18aad None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:53.137165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:53.163115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:53.171600 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:53.172545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:53.173011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:53.173292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:53.174021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:53.174334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:53.175712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:53.188804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 538/1075] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:53.251401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64a8a2-5bd4-466a-8192-7d40a1ce6afa None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:53.296049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:53.296758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:53.297041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:53.297315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:53.298128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:53.298440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:53.299922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892149-4605-44b1-ac31-109249e9fc2f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:53.319359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 538/1076] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:53.320466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:53.413817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:53.431528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37ead7-750c-4b2e-a8e2-10daf31257b6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:53.466102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298b6e2-2cf3-4d26-bdcf-275d5dd8e1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:53.467275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b6e2-2cf3-4d26-bdcf-275d5dd8e1ed None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:53.467704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b6e2-2cf3-4d26-bdcf-275d5dd8e1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:53.468162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b6e2-2cf3-4d26-bdcf-275d5dd8e1ed None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/14dd1de1fe914f319017818e571732de/roles/f39bdab856724510a905efb68870ac32` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:53.470508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298b6e2-2cf3-4d26-bdcf-275d5dd8e1ed None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=14dd1de1fe914f319017818e571732de, role_id=f39bdab856724510a905efb68870ac32)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:53.472213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298b6e2-2cf3-4d26-bdcf-275d5dd8e1ed None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:53.487514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 539/1077] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:53 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/14dd1de1fe914f319017818e571732de/roles/f39bdab856724510a905efb68870ac32 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:48:54.147696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c4d81-a6a1-4d2e-854c-e0248a8a0426 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:54.148298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c4d81-a6a1-4d2e-854c-e0248a8a0426 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:54.148298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c4d81-a6a1-4d2e-854c-e0248a8a0426 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:54.149694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 539/1078] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:54.155261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:54.155617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:54.155994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:54.173223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:54.174398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:54.242531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 540/1079] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:48:54.265630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c4d81-a6a1-4d2e-854c-e0248a8a0426 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:54.353458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:54.373218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c4d81-a6a1-4d2e-854c-e0248a8a0426 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:54.407371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:54.408123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:54.408403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:54.408675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:54.419772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:54.427302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:54.428401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:54.429284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 540/1080] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:54.434790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:54.512781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:54.538423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc3533-6a4e-47d9-8ac9-444bee68d7dd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:54.573992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:54.574664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:54.574945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:54.575221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:54.576010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:54.576319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:54.577698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:54.586434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 541/1081] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:54.592298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:54.690765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf15bc-798e-4552-b89d-5e813014e542 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:54.725347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:54.726219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:54.726492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:54.726756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] PATH_INFO: `/v3/users/5e61f06f1d684d1aba97c2d85c09839e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:54.728008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] RBAC: Authorizing `identity:get_user(user_id=5e61f06f1d684d1aba97c2d85c09839e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:54.729424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:54.731079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 541/1082] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:48:54 2022] GET /identity/v3/users/5e61f06f1d684d1aba97c2d85c09839e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:54.737107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:54.840091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc357a-2e1c-4bd7-8c9f-8f95248ba6a9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:54.873946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead660ba-182c-4f49-a182-05c4b65528d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:54.874801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead660ba-182c-4f49-a182-05c4b65528d9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:54.875079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead660ba-182c-4f49-a182-05c4b65528d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:54.875354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead660ba-182c-4f49-a182-05c4b65528d9 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:54.876595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead660ba-182c-4f49-a182-05c4b65528d9 None admin] RBAC: Authorizing `identity:get_project(project_id=dbb51914ad5845c3a446c74365f1fd33)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:54.878031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead660ba-182c-4f49-a182-05c4b65528d9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:54.879730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 542/1083] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:48:54 2022] GET /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:54.886095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:54.898920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:54.986249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e927fc-2196-4e50-b550-c0ae86ecd435 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:55.022117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9ca2-e05c-4619-bb23-5c0429ccbbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:55.023007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9ca2-e05c-4619-bb23-5c0429ccbbb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:55.023286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9ca2-e05c-4619-bb23-5c0429ccbbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:55.023557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9ca2-e05c-4619-bb23-5c0429ccbbb4 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33/users/5e61f06f1d684d1aba97c2d85c09839e/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:55.025643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9ca2-e05c-4619-bb23-5c0429ccbbb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbb51914ad5845c3a446c74365f1fd33, user_id=5e61f06f1d684d1aba97c2d85c09839e, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:55.027083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9ca2-e05c-4619-bb23-5c0429ccbbb4 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:55.041894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 542/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:48:54 2022] PUT /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33/users/5e61f06f1d684d1aba97c2d85c09839e/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:48:55.461618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696863c-be74-464f-841f-db3ae040a198 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:55.462211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696863c-be74-464f-841f-db3ae040a198 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:55.462211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696863c-be74-464f-841f-db3ae040a198 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:55.463575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 543/1085] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:55.468971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:55.469432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:55.469525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:55.488206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:55.489377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:55.553775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:55.558638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 543/1086] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:55.581810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696863c-be74-464f-841f-db3ae040a198 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:55.592718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:55.693440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696863c-be74-464f-841f-db3ae040a198 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:55.729950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:55.730775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:55.731056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:55.731327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:55.742674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:55.750836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:55.751860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:55.752924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 544/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 09:48:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:55.758852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:55.862786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fe652-d68c-4b3a-b90c-c8e9f472d6b8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:55.899331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:55.900032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:55.900315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:55.900589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:55.901335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:55.901642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:55.903024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:55.915964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 544/1088] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:48:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:55.917124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:55.990555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:56.028169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82a789-033d-42b2-9ba5-06d07fbdc766 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:56.063346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:56.064181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:56.064442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:56.064737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:56.075672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:56.077170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:56.094731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:56.095637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:56.101530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:56.101530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:56.101530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:56.101530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:56.101530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 545/1089] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:56.101975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:56.167054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:56.206977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6eb08-2894-406e-9c0d-32977e69ea44 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:56.244719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:56.245389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:56.245669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:56.245936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:56.246663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:56.246969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:56.248502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:56.260483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 545/1090] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:56.266495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:56.373224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c65635-b5c2-43ad-b83f-1b7be97a5ef3 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:56.390149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:56.407354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:56.408206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:56.408487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:56.408761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:56.416970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:56.418462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:48:56.432147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:56.433095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 09:48:56.434024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:56.436252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 546/1091] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 09:48:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:56.439435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:56.540240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafb1b-b9f1-42e1-b1e5-d2e688b5ffc0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:56.563066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:56.575928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:56.576626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:56.576901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:56.577173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:56.578010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:56.578317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:56.579756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:56.597819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 546/1092] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 09:48:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:56.603913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:56.713269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048a043-b455-4118-a494-6f19e13e1ef2 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:56.747893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd36ee-3de4-427d-8740-23eca54ab2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:56.748540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd36ee-3de4-427d-8740-23eca54ab2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:56.748824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd36ee-3de4-427d-8740-23eca54ab2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:56.749101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd36ee-3de4-427d-8740-23eca54ab2f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:56.795261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:56.816852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd36ee-3de4-427d-8740-23eca54ab2f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:56.818343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd36ee-3de4-427d-8740-23eca54ab2f9 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:56.918657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 547/1093] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:56.924374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.026988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874a407-027c-4497-b183-a804597a7c82 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.027181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874a407-027c-4497-b183-a804597a7c82 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.027527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874a407-027c-4497-b183-a804597a7c82 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.029724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 548/1094] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:48:57.030237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79458dc5-773a-4561-96e8-b54a35657739 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.036767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.037050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.037328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.054402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:48:57.055628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:48:57.069013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.069806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.070091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.070371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.080732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.082197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:48:57.100807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:57.101738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:57.102634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:48:57.102634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:48:57.102634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:48:57.102634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:57.102634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 547/1095] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:57.107749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.128319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 549/1096] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:48:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:48:57.137181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.152791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.164041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.214925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a42a-6d5f-456f-85bd-654d8c07970a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.250536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.251275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.251552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.251821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.252565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:57.252866 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.254239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.255652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc494b3-192f-409c-90bf-fa45f5778322 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.265370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 548/1097] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:57.271149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.292902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.293904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.293904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.294110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.304447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:48:57.311911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:57.313042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:48:57.314365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 550/1098] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:48:57.319142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.368436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.377838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd163bc-2438-4961-af1c-ff508ffa9ceb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.386100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.411722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.412590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.412870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.413164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.419228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe911f-9f50-4667-8557-062af8621ffa None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.419686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.421178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:48:57.435294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:48:57.436263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:48:57.437412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 549/1099] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:48:57.442543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.451835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.452523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.452803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.453081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.453849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:57.454153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.455515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.464270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 551/1100] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:57.470032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.543340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5842d5-ac46-404a-a961-48317d57e4cd None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.570651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aaed5-19d4-4f71-b67e-37bf02200a46 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.577153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.577815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.578092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.578366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.579140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:57.579441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.580933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.592959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 550/1101] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:57.598295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.599115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.605160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.606091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.606376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.606652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] PATH_INFO: `/v3/users/5e61f06f1d684d1aba97c2d85c09839e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.607928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] RBAC: Authorizing `identity:get_user(user_id=5e61f06f1d684d1aba97c2d85c09839e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.609422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.611142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 552/1102] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/users/5e61f06f1d684d1aba97c2d85c09839e => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:57.617150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:57.617305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.709224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9eef0-23f1-4b6f-9a9a-416f3a0042c0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.719565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9ff5c-25e2-4db1-9b4e-07be894115cc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:57.743682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.744481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.744783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.745050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.745879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] RBAC: Adding query filter params (role.id=f39bdab856724510a905efb68870ac32, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=14dd1de1fe914f319017818e571732de) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:57.746183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.747628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.752348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:57.752996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:57.753278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:57.753550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:57.753643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 551/1103] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/role_assignments?user.id=14dd1de1fe914f319017818e571732de&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=f39bdab856724510a905efb68870ac32 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:48:57.822441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:57.826522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:57.828395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:57.935487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 553/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:48:57.941434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:58.024648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:58.050824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6fd54-1989-4cec-b402-b7a1b6d2fba8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:58.084572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a7737-8528-4b77-a494-ffd288b2eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:58.085715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a7737-8528-4b77-a494-ffd288b2eb82 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:58.085989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a7737-8528-4b77-a494-ffd288b2eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:58.086252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a7737-8528-4b77-a494-ffd288b2eb82 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:58.087448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a7737-8528-4b77-a494-ffd288b2eb82 None admin] RBAC: Authorizing `identity:get_project(project_id=dbb51914ad5845c3a446c74365f1fd33)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:58.088920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a7737-8528-4b77-a494-ffd288b2eb82 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:58.090895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 552/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:48:57 2022] GET /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:48:58.097038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:48:58.199027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:48:58.202971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cebd7-3933-4934-9793-ab99cc3044ae None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:48:58.237276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:48:58.237957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:48:58.238239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:48:58.238500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:48:58.239413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=dbb51914ad5845c3a446c74365f1fd33, user.id=5e61f06f1d684d1aba97c2d85c09839e) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:48:58.239713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:48:58.241179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21d8e6-4202-4867-8629-12af2dea5f15 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:48:58.247297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 554/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:48:58 2022] GET /identity/v3/role_assignments?user.id=5e61f06f1d684d1aba97c2d85c09839e&scope.project.id=dbb51914ad5845c3a446c74365f1fd33&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:00.283512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a917e-3c50-4dc7-9a94-49ba157ba374 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:00.284447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a917e-3c50-4dc7-9a94-49ba157ba374 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:00.284447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a917e-3c50-4dc7-9a94-49ba157ba374 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:00.286601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 553/1107] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:00.292315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:00.292664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:00.292750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:00.310125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:00.311310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:00.379413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 555/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:00.405818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a917e-3c50-4dc7-9a94-49ba157ba374 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:00.520143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a917e-3c50-4dc7-9a94-49ba157ba374 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:00.557004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8000c9-78f2-4c21-951e-66a96ef0583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:00.557722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8000c9-78f2-4c21-951e-66a96ef0583a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:00.558005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8000c9-78f2-4c21-951e-66a96ef0583a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:00.558277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8000c9-78f2-4c21-951e-66a96ef0583a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:00.559516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8000c9-78f2-4c21-951e-66a96ef0583a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:00.561001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8000c9-78f2-4c21-951e-66a96ef0583a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:00.562644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 554/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:00.569184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:00.667023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:00.679213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f7e87-5a67-4fcb-94f6-1e1a8f64c03d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:00.715815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e725f2-4b36-4991-9bad-b5d2ce2a9fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:00.716516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e725f2-4b36-4991-9bad-b5d2ce2a9fe8 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:00.716799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e725f2-4b36-4991-9bad-b5d2ce2a9fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:00.717076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e725f2-4b36-4991-9bad-b5d2ce2a9fe8 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:00.717790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e725f2-4b36-4991-9bad-b5d2ce2a9fe8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:00.719242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e725f2-4b36-4991-9bad-b5d2ce2a9fe8 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:00.740713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 556/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:49:00 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:49:02.804449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67356d43-7b63-46b0-80ab-27f204fb6ca3 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:02.805033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67356d43-7b63-46b0-80ab-27f204fb6ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:02.805033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67356d43-7b63-46b0-80ab-27f204fb6ca3 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:02.806391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 555/1111] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:02.811997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:02.812402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:02.812565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:02.830669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:02.831862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:02.900873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 557/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:02.924744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67356d43-7b63-46b0-80ab-27f204fb6ca3 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:02.942799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:03.034636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67356d43-7b63-46b0-80ab-27f204fb6ca3 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:03.069225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:03.070017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:03.070301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:03.070579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] PATH_INFO: `/v3/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:03.072094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] RBAC: Authorizing `identity:get_role(role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:03.073475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:03.074850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 556/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:02 2022] GET /identity/v3/roles/805a0b1c9f554aff8a16769124f939eb => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:03.081058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:03.164688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:03.190328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea572850-08f8-4596-b043-7802a358f8dd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:03.226169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:03.227044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:03.227321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:03.227592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] PATH_INFO: `/v3/users/6d609920a9bf40f2b90a6944b01b2eff` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:03.241394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] RBAC: Authorizing `identity:get_user(user_id=6d609920a9bf40f2b90a6944b01b2eff)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:03.242838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:03.244530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 558/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:03 2022] GET /identity/v3/users/6d609920a9bf40f2b90a6944b01b2eff => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:03.250884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:03.338725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:03.359443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cbd68-073a-45d9-a814-5d64df4d22f5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:03.395109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:03.395800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:03.396115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:03.396398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:03.467835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:03.469308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:03.549784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:03.572346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 557/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:03.579257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:03.688484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf891ee-3368-4bce-990c-a6b7ff8db2cc None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:03.723840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b4674-04ce-434d-9dbc-b53d0fb2a0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:03.724717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b4674-04ce-434d-9dbc-b53d0fb2a0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:03.724981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b4674-04ce-434d-9dbc-b53d0fb2a0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:03.725270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b4674-04ce-434d-9dbc-b53d0fb2a0a2 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:03.726472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b4674-04ce-434d-9dbc-b53d0fb2a0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=dbb51914ad5845c3a446c74365f1fd33)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:03.727893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b4674-04ce-434d-9dbc-b53d0fb2a0a2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:03.729617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 559/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:03 2022] GET /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:03.739600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:03.849587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9811ea4-340a-4690-9ed1-ade6e353dfc5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:03.888291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:03.888932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:03.889207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:03.889478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:03.890303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] RBAC: Adding query filter params (role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=dbb51914ad5845c3a446c74365f1fd33, user.id=6d609920a9bf40f2b90a6944b01b2eff) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:03.890608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:03.892053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb7f9e-2348-4d2d-99b6-4bbe11f8bbc9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:03.897753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 558/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:49:03 2022] GET /identity/v3/role_assignments?user.id=6d609920a9bf40f2b90a6944b01b2eff&scope.project.id=dbb51914ad5845c3a446c74365f1fd33&role.id=805a0b1c9f554aff8a16769124f939eb => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:05.937038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125a6bc-8b7c-4b4e-8d93-31834a5386ba None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:05.937946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125a6bc-8b7c-4b4e-8d93-31834a5386ba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:05.937946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125a6bc-8b7c-4b4e-8d93-31834a5386ba None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:05.940147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 560/1118] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:05.948907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:05.949618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:05.949796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:05.970451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:05.971647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:06.015119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:06.045385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 559/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:06.069307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125a6bc-8b7c-4b4e-8d93-31834a5386ba None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:06.075697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:06.187854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125a6bc-8b7c-4b4e-8d93-31834a5386ba None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:06.225506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:06.226367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:06.226642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:06.226910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] PATH_INFO: `/v3/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:06.228508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] RBAC: Authorizing `identity:get_role(role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:06.229888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:06.231258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 561/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:06 2022] GET /identity/v3/roles/805a0b1c9f554aff8a16769124f939eb => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:06.240112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:06.347883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f487c-b38d-435a-b24a-596fbde431e8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:06.383139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:06.384031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:06.384309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:06.384562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] PATH_INFO: `/v3/users/6d609920a9bf40f2b90a6944b01b2eff` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:06.385772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] RBAC: Authorizing `identity:get_user(user_id=6d609920a9bf40f2b90a6944b01b2eff)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:06.387223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:06.388912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 560/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:06 2022] GET /identity/v3/users/6d609920a9bf40f2b90a6944b01b2eff => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:06.395651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:06.479295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:06.511892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa75c3e-8868-47e5-936c-a9954a26137d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:06.549507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5edce-e2b7-4bf9-982f-8f76b2819236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:06.550324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5edce-e2b7-4bf9-982f-8f76b2819236 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:06.550637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5edce-e2b7-4bf9-982f-8f76b2819236 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:06.550915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5edce-e2b7-4bf9-982f-8f76b2819236 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:06.552385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5edce-e2b7-4bf9-982f-8f76b2819236 None admin] RBAC: Authorizing `identity:get_project(project_id=dbb51914ad5845c3a446c74365f1fd33)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:06.553888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5edce-e2b7-4bf9-982f-8f76b2819236 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:06.555567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 562/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:06 2022] GET /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:06.565520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:06.622068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:06.680962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f9f7-e93a-4d68-890a-1477f8bc91f5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:06.716036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14734e-366d-4f18-9957-e118235d47e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:06.716919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14734e-366d-4f18-9957-e118235d47e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:06.717192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14734e-366d-4f18-9957-e118235d47e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:06.717454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14734e-366d-4f18-9957-e118235d47e4 None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33/users/6d609920a9bf40f2b90a6944b01b2eff/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:06.720069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14734e-366d-4f18-9957-e118235d47e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbb51914ad5845c3a446c74365f1fd33, user_id=6d609920a9bf40f2b90a6944b01b2eff, role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:06.721509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14734e-366d-4f18-9957-e118235d47e4 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:06.741397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 561/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:49:06 2022] PUT /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33/users/6d609920a9bf40f2b90a6944b01b2eff/roles/805a0b1c9f554aff8a16769124f939eb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:49:08.728668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d95f3-1635-4e3f-bfe7-4e53f46fb231 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:08.729561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d95f3-1635-4e3f-bfe7-4e53f46fb231 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:08.729561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d95f3-1635-4e3f-bfe7-4e53f46fb231 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:08.731761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 563/1124] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:08.738160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:08.739049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:08.739205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:08.757298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:08.758474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:08.826891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 562/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:08.851301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d95f3-1635-4e3f-bfe7-4e53f46fb231 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:08.965493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d95f3-1635-4e3f-bfe7-4e53f46fb231 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:09.000712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:09.001506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:09.001783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:09.002057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] PATH_INFO: `/v3/roles/805a0b1c9f554aff8a16769124f939eb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:09.003535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] RBAC: Authorizing `identity:get_role(role_id=805a0b1c9f554aff8a16769124f939eb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:09.005204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:09.006569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 564/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:08 2022] GET /identity/v3/roles/805a0b1c9f554aff8a16769124f939eb => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:09.016189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:09.056215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:09.128647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dd14f-e3fe-49fd-974a-c50387c5108b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:09.163491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:09.164378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:09.164659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:09.164930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] PATH_INFO: `/v3/users/6d609920a9bf40f2b90a6944b01b2eff` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:09.166224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] RBAC: Authorizing `identity:get_user(user_id=6d609920a9bf40f2b90a6944b01b2eff)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:09.167646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:09.169312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 563/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:09 2022] GET /identity/v3/users/6d609920a9bf40f2b90a6944b01b2eff => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:09.175127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:09.186950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:09.296504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b09cbe-e251-4094-b023-5cd2fe273337 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:09.331529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:09.332209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:09.332450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:09.332900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:09.401115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:09.402568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:09.411891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:09.508616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 565/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:09.518156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:09.620822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:09.624113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385bbf-8dfa-45e6-8bda-64bc3bee67f5 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:09.661255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851fb0c-c068-4083-a913-64683fe2bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:09.662113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fb0c-c068-4083-a913-64683fe2bb1d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:09.662395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fb0c-c068-4083-a913-64683fe2bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:09.662665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fb0c-c068-4083-a913-64683fe2bb1d None admin] PATH_INFO: `/v3/projects/dbb51914ad5845c3a446c74365f1fd33` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:09.663993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851fb0c-c068-4083-a913-64683fe2bb1d None admin] RBAC: Authorizing `identity:get_project(project_id=dbb51914ad5845c3a446c74365f1fd33)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:09.665436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851fb0c-c068-4083-a913-64683fe2bb1d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:09.667124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 564/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:09 2022] GET /identity/v3/projects/dbb51914ad5845c3a446c74365f1fd33 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:09.673890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:09.781285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae676ca7-85ae-427f-8f9d-c9f45b7aa154 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:09.791069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:09.820382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:09.821020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:09.821299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:09.821570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:09.822396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] RBAC: Adding query filter params (role.id=805a0b1c9f554aff8a16769124f939eb, scope.project.id=dbb51914ad5845c3a446c74365f1fd33, user.id=6d609920a9bf40f2b90a6944b01b2eff) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:09.822713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:09.824158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c98d6-e0ed-406f-a706-c067197532a2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:09.830064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 566/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:49:09 2022] GET /identity/v3/role_assignments?user.id=6d609920a9bf40f2b90a6944b01b2eff&scope.project.id=dbb51914ad5845c3a446c74365f1fd33&role.id=805a0b1c9f554aff8a16769124f939eb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:11.868865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19834ef3-df86-4a5d-a8fd-55957f7fc8b9 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:11.869425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19834ef3-df86-4a5d-a8fd-55957f7fc8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:11.869425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19834ef3-df86-4a5d-a8fd-55957f7fc8b9 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:11.870802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 565/1131] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:11.878648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:11.879069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:11.879255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:11.901755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:11.902953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:11.977370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 567/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:12.001617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19834ef3-df86-4a5d-a8fd-55957f7fc8b9 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:12.110759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19834ef3-df86-4a5d-a8fd-55957f7fc8b9 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:12.146616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cd3f8-3d12-4bc7-9291-4559c6d13d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:12.147320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd3f8-3d12-4bc7-9291-4559c6d13d16 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:12.147599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd3f8-3d12-4bc7-9291-4559c6d13d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:12.147905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd3f8-3d12-4bc7-9291-4559c6d13d16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:12.149112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd3f8-3d12-4bc7-9291-4559c6d13d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:12.150553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd3f8-3d12-4bc7-9291-4559c6d13d16 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:12.152206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 566/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:12.158264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:12.282909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661a46d-2d54-46ae-857c-7b98bf67aa2b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:12.284550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:12.322495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f09d7-48e9-41e9-a758-7bdca040cef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:12.323154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f09d7-48e9-41e9-a758-7bdca040cef3 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:12.323437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f09d7-48e9-41e9-a758-7bdca040cef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:12.323712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f09d7-48e9-41e9-a758-7bdca040cef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:12.324457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f09d7-48e9-41e9-a758-7bdca040cef3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:12.325904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f09d7-48e9-41e9-a758-7bdca040cef3 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:12.361563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 568/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:49:12 2022] POST /identity/v3/projects => generated 311 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:49:14.421285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0727f-c2ae-44c4-8800-03b886924413 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:14.421839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0727f-c2ae-44c4-8800-03b886924413 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:14.421839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0727f-c2ae-44c4-8800-03b886924413 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:14.423935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 567/1135] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:14.432710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:14.433106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:14.433326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:14.454938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:14.456153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:14.530696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 569/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:14.556019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0727f-c2ae-44c4-8800-03b886924413 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:14.603471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:14.671746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0727f-c2ae-44c4-8800-03b886924413 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:14.711847 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1aa7a-0e95-45f3-8196-b46aeca679f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:14.712734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1aa7a-0e95-45f3-8196-b46aeca679f7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:14.713010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1aa7a-0e95-45f3-8196-b46aeca679f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:14.713275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1aa7a-0e95-45f3-8196-b46aeca679f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:14.714569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1aa7a-0e95-45f3-8196-b46aeca679f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:14.716049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1aa7a-0e95-45f3-8196-b46aeca679f7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:14.717742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 568/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:49:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:14.726952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:14.812913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:14.833229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5c288-8695-4567-a4c7-b1467243f0cd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:14.868210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5f60-2b61-4560-bcbc-89f2bca4f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:14.868860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5f60-2b61-4560-bcbc-89f2bca4f289 None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:14.869140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5f60-2b61-4560-bcbc-89f2bca4f289 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:14.869413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5f60-2b61-4560-bcbc-89f2bca4f289 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:14.870131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5f60-2b61-4560-bcbc-89f2bca4f289 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:14.871583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5f60-2b61-4560-bcbc-89f2bca4f289 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:14.893626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 570/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:49:14 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:49:16.918193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851ad5c-7eee-4e50-8568-522c18dd24ca None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:16.919083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851ad5c-7eee-4e50-8568-522c18dd24ca None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:16.919083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851ad5c-7eee-4e50-8568-522c18dd24ca None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:16.921329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 569/1139] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:16.928878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:16.929304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:16.929447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:16.957427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:16.958608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:17.030124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 571/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:17.057265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851ad5c-7eee-4e50-8568-522c18dd24ca None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:17.164630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851ad5c-7eee-4e50-8568-522c18dd24ca None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:17.166924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:17.199233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:17.199965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:17.200241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:17.200513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:17.212036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:49:17.219466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:17.220690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:49:17.221610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 570/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:17.229438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:17.304978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:17.338448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92057d-6ae0-437a-840c-81014fd64691 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:17.374844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:17.375508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:17.375787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:17.376096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:17.376849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:17.377161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:17.378542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:17.387126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 572/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:17.393818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:17.511078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96915caf-ebaf-494d-97d7-b472b6b718af None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:17.545993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:17.546857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:17.547134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:17.547399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] PATH_INFO: `/v3/users/dcbca89b6570475ba99594d1a7eba4f2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:17.561253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] RBAC: Authorizing `identity:get_user(user_id=dcbca89b6570475ba99594d1a7eba4f2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:17.562700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:17.564380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 571/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:17 2022] GET /identity/v3/users/dcbca89b6570475ba99594d1a7eba4f2 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:17.570400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:17.675545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9b95-1a02-44c4-ae2c-90c873e9c6a7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:17.704919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:17.710979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:17.711617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:17.711919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:17.712181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:17.779692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:17.781098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:17.881201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 573/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:17.887281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:17.914642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:17.994744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d512-2a8f-4b98-a9d3-6ba8b6c715e8 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:18.029030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345460c5-cd0f-49c8-9716-859fa9757837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:18.029951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345460c5-cd0f-49c8-9716-859fa9757837 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:18.030161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345460c5-cd0f-49c8-9716-859fa9757837 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:18.030407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345460c5-cd0f-49c8-9716-859fa9757837 None admin] PATH_INFO: `/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:18.031637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345460c5-cd0f-49c8-9716-859fa9757837 None admin] RBAC: Authorizing `identity:get_project(project_id=b41aaa4d48e04bc98a3079bdb57d17e4)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:18.033110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345460c5-cd0f-49c8-9716-859fa9757837 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:18.034822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 572/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:17 2022] GET /identity/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:18.044239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:18.074983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:18.143385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee86f4-b5a4-4b74-8f08-7c08b8188c57 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:18.186308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:18.187277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:18.187668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:18.188103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:18.189330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=b41aaa4d48e04bc98a3079bdb57d17e4, user.id=dcbca89b6570475ba99594d1a7eba4f2) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:18.189778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:18.191975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df053d-fa6e-4047-b335-e48b59490dcd None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:18.197873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 574/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:49:18 2022] GET /identity/v3/role_assignments?user.id=dcbca89b6570475ba99594d1a7eba4f2&scope.project.id=b41aaa4d48e04bc98a3079bdb57d17e4&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:20.168289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c326-c3bc-4c5f-9959-0118922edbc1 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:20.168874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c326-c3bc-4c5f-9959-0118922edbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:20.168874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c326-c3bc-4c5f-9959-0118922edbc1 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:20.170353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 573/1147] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:20.175509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:20.176029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:20.176262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:20.193145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:20.194293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:20.262617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 575/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:20.286318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c326-c3bc-4c5f-9959-0118922edbc1 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:20.397381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:20.409946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c326-c3bc-4c5f-9959-0118922edbc1 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:20.445222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:20.445939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:20.446218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:20.446478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:20.457213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:49:20.465066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:20.466113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:49:20.467328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 574/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:49:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:20.475075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:20.503626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:20.579557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b5799-d1d0-464f-a40a-ff51de2591f6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:20.615556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:20.616241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:20.616519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:20.616793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:20.617546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:20.617851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:20.619242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:20.628502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 576/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:49:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:20.635670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:20.749168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5f1b8-e93e-46e5-ad03-4835607a7f7c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:20.786445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:20.787308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:20.787585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:20.787854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] PATH_INFO: `/v3/users/dcbca89b6570475ba99594d1a7eba4f2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:20.789180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] RBAC: Authorizing `identity:get_user(user_id=dcbca89b6570475ba99594d1a7eba4f2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:20.790613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:20.792262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 575/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:20 2022] GET /identity/v3/users/dcbca89b6570475ba99594d1a7eba4f2 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:20.802329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:20.892659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:20.912984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680080-96ef-4bdc-82a9-2d0ae5fb1917 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:20.951109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775e8f-f202-4ffc-936a-bc62578a8a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:20.952055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775e8f-f202-4ffc-936a-bc62578a8a30 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:20.952335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775e8f-f202-4ffc-936a-bc62578a8a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:20.952604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775e8f-f202-4ffc-936a-bc62578a8a30 None admin] PATH_INFO: `/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:20.953790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775e8f-f202-4ffc-936a-bc62578a8a30 None admin] RBAC: Authorizing `identity:get_project(project_id=b41aaa4d48e04bc98a3079bdb57d17e4)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:20.955204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775e8f-f202-4ffc-936a-bc62578a8a30 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:20.956916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 577/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:20 2022] GET /identity/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:20.964357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:20.993267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:21.079170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0828b-938a-4693-a724-fa01d9f00298 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:21.115390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373ad7-7e8d-4d78-af41-54f12d95115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:21.116302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373ad7-7e8d-4d78-af41-54f12d95115f None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:21.116576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373ad7-7e8d-4d78-af41-54f12d95115f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:21.116849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373ad7-7e8d-4d78-af41-54f12d95115f None admin] PATH_INFO: `/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4/users/dcbca89b6570475ba99594d1a7eba4f2/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:21.118808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5373ad7-7e8d-4d78-af41-54f12d95115f None admin] RBAC: Authorizing `identity:create_grant(project_id=b41aaa4d48e04bc98a3079bdb57d17e4, user_id=dcbca89b6570475ba99594d1a7eba4f2, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:21.120280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5373ad7-7e8d-4d78-af41-54f12d95115f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:21.135844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 576/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:49:20 2022] PUT /identity/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4/users/dcbca89b6570475ba99594d1a7eba4f2/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:49:23.121577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae4d4a-3eab-44fa-be0c-3e056e061dbe None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:23.122429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae4d4a-3eab-44fa-be0c-3e056e061dbe None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:23.122429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae4d4a-3eab-44fa-be0c-3e056e061dbe None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:23.124741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 578/1154] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:23.133908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:23.134343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:23.134469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:23.156329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:23.157508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:23.233516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 577/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:23.257795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae4d4a-3eab-44fa-be0c-3e056e061dbe None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:23.371681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae4d4a-3eab-44fa-be0c-3e056e061dbe None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:23.406851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:23.407628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:23.407917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:23.408200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:23.422048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:49:23.429552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:23.430615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:49:23.431807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 579/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:23.439804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:23.451120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:23.553788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d651cc-01c9-46f2-89bb-81c6abd047e2 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:23.595721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:23.596820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:23.597261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:23.597728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:23.598493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:23.598794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:23.600198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:23.608860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 578/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:49:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:23.617278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:23.646792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:23.743680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785400f-0e7e-4265-9864-ab006bac58af None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:23.781198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:23.782085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:23.782366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:23.782604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] PATH_INFO: `/v3/users/dcbca89b6570475ba99594d1a7eba4f2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:23.783962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] RBAC: Authorizing `identity:get_user(user_id=dcbca89b6570475ba99594d1a7eba4f2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:23.785465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:23.787208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 580/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:23 2022] GET /identity/v3/users/dcbca89b6570475ba99594d1a7eba4f2 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:23.793469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:23.867280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:23.899007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095da522-5093-4f18-a83a-2cb544066164 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:23.934781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:23.935420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:23.935701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:23.936005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:24.006752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:24.008224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:24.080001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:24.116323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 579/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:24.122738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:24.213534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:24.230717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4471cc5-a079-46f6-8093-745e960e1b4e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:24.264864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397be225-9578-425a-b41c-9300834fa10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:24.265715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397be225-9578-425a-b41c-9300834fa10c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:24.265994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397be225-9578-425a-b41c-9300834fa10c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:24.266264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397be225-9578-425a-b41c-9300834fa10c None admin] PATH_INFO: `/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:24.267455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397be225-9578-425a-b41c-9300834fa10c None admin] RBAC: Authorizing `identity:get_project(project_id=b41aaa4d48e04bc98a3079bdb57d17e4)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:24.268942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397be225-9578-425a-b41c-9300834fa10c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:24.270667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 581/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:24 2022] GET /identity/v3/projects/b41aaa4d48e04bc98a3079bdb57d17e4 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:24.277288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:24.393525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a04-813c-4fd3-b64f-e3e66142638e None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:24.430037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:24.430684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:24.430965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:24.431241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:24.432099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=b41aaa4d48e04bc98a3079bdb57d17e4, user.id=dcbca89b6570475ba99594d1a7eba4f2) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:24.432409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:24.433829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e2c6e-1f44-4825-9507-2efb21ebeeaf None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:24.439742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 580/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:49:24 2022] GET /identity/v3/role_assignments?user.id=dcbca89b6570475ba99594d1a7eba4f2&scope.project.id=b41aaa4d48e04bc98a3079bdb57d17e4&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:26.483515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0688a3-1a43-4f40-a1dd-14e8b2ccf582 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:26.484119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0688a3-1a43-4f40-a1dd-14e8b2ccf582 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:26.484119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0688a3-1a43-4f40-a1dd-14e8b2ccf582 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:26.486399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 582/1162] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:26.494647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:26.495233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:26.495233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:26.517133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:26.518336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:26.550694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:26.603573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 581/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:26.632058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0688a3-1a43-4f40-a1dd-14e8b2ccf582 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:26.759110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0688a3-1a43-4f40-a1dd-14e8b2ccf582 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:26.803637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:26.804998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:26.804998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:26.804998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:26.828096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:26.880844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:26.882349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:26.991062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 583/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:27.001139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:27.134198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3652e-3727-483f-a62f-df77e357d277 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:27.182973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:27.183779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:27.184086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:27.184353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:27.192021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:27.193481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-c2221ae9-7972-49c0-9df0-7aa66de3f831 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:27.211845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:27.213072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:49:27.214181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 582/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:49:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:27.221996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:27.297385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:27.350788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3fa83-d7e6-4f8d-8904-edce9c7c5765 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:27.386173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:27.386750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:27.387026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:27.387302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:27.388058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:27.388366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:27.389740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90200f3-1ed1-4880-8430-970a7f0df8d5 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:27.396951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 584/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 09:49:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:29.391396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31b0df-a831-4932-a064-461d01490d0b None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:29.392387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31b0df-a831-4932-a064-461d01490d0b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:29.392387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31b0df-a831-4932-a064-461d01490d0b None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:29.394759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 583/1167] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:29.400175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa635b5-f711-464c-a981-20328553460f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:29.400650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa635b5-f711-464c-a981-20328553460f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:29.400869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa635b5-f711-464c-a981-20328553460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:29.418772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8aa635b5-f711-464c-a981-20328553460f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:29.419962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8aa635b5-f711-464c-a981-20328553460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:29.488019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 585/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:29.511592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31b0df-a831-4932-a064-461d01490d0b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:29.547836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:29.626933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31b0df-a831-4932-a064-461d01490d0b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:29.662190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e4113-1bde-4571-9154-09989adde7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:29.662840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e4113-1bde-4571-9154-09989adde7e9 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:29.663125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e4113-1bde-4571-9154-09989adde7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:29.663408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e4113-1bde-4571-9154-09989adde7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:29.664175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e4113-1bde-4571-9154-09989adde7e9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:29.665549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e4113-1bde-4571-9154-09989adde7e9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:29.686183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 584/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Jun 30 09:49:29 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:49:31.733915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e068d7d-3f2e-44d2-94a0-6beed5c226ca None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:31.734465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e068d7d-3f2e-44d2-94a0-6beed5c226ca None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:31.734465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e068d7d-3f2e-44d2-94a0-6beed5c226ca None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:31.736625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 586/1170] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:31.745466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:31.745876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:31.746083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:31.766352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:31.767531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:31.850966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 585/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:31.879405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e068d7d-3f2e-44d2-94a0-6beed5c226ca None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:31.943765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:32.005388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e068d7d-3f2e-44d2-94a0-6beed5c226ca None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:32.040849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cc516-9fb5-470e-a20d-e820bc958ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:32.041708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cc516-9fb5-470e-a20d-e820bc958ffa None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:32.041988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cc516-9fb5-470e-a20d-e820bc958ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:32.042267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cc516-9fb5-470e-a20d-e820bc958ffa None admin] PATH_INFO: `/v3/domains/8207d33f82e140958369a295fd9a657f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:32.053946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cc516-9fb5-470e-a20d-e820bc958ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=8207d33f82e140958369a295fd9a657f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:32.055418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cc516-9fb5-470e-a20d-e820bc958ffa None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:32.057094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 587/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Jun 30 09:49:31 2022] GET /identity/v3/domains/8207d33f82e140958369a295fd9a657f => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:32.066862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:32.084316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:32.176270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0668074-d12f-460c-94ef-986673a851f9 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:32.213084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a19109-bd6a-4d10-9453-4ee055201865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:32.213733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a19109-bd6a-4d10-9453-4ee055201865 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:32.214020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a19109-bd6a-4d10-9453-4ee055201865 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:32.214293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a19109-bd6a-4d10-9453-4ee055201865 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:32.215019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a19109-bd6a-4d10-9453-4ee055201865 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:32.216512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a19109-bd6a-4d10-9453-4ee055201865 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:32.245934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 586/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:49:32 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 09:49:34.272516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67d3b-d315-4253-bbdc-cee68112c4a6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:34.273101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67d3b-d315-4253-bbdc-cee68112c4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:34.273101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67d3b-d315-4253-bbdc-cee68112c4a6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:34.274466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 588/1174] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:34.281016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:34.281739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:34.281886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:34.306404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:34.307604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:34.371425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:34.380355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 587/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:34.410592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67d3b-d315-4253-bbdc-cee68112c4a6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:34.518278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67d3b-d315-4253-bbdc-cee68112c4a6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:34.526865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:34.555610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c208580-e317-437c-b95e-1b4be331c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:34.556497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208580-e317-437c-b95e-1b4be331c67b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:34.556774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208580-e317-437c-b95e-1b4be331c67b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:34.557046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208580-e317-437c-b95e-1b4be331c67b None admin] PATH_INFO: `/v3/domains/8207d33f82e140958369a295fd9a657f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:34.558240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c208580-e317-437c-b95e-1b4be331c67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=8207d33f82e140958369a295fd9a657f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:34.559679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c208580-e317-437c-b95e-1b4be331c67b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:34.561323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 589/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Jun 30 09:49:34 2022] GET /identity/v3/domains/8207d33f82e140958369a295fd9a657f => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:34.567935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:34.684801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc5df1-b395-4e96-b63c-075125618725 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:34.721331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c259d0a-01ca-474f-b9ba-45da3b4ea2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:34.721989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c259d0a-01ca-474f-b9ba-45da3b4ea2c7 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:34.722322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c259d0a-01ca-474f-b9ba-45da3b4ea2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:34.722596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c259d0a-01ca-474f-b9ba-45da3b4ea2c7 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:34.723307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c259d0a-01ca-474f-b9ba-45da3b4ea2c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:34.724775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c259d0a-01ca-474f-b9ba-45da3b4ea2c7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:34.751638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 588/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:49:34 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:49:36.795023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e7f9-1d75-4fcf-8cab-68e4699fb1c0 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:36.795950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e7f9-1d75-4fcf-8cab-68e4699fb1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:36.795950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e7f9-1d75-4fcf-8cab-68e4699fb1c0 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:36.798208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 590/1178] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:36.804633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:36.805345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:36.805501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:36.826871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:36.828086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:36.880561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:36.902369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 589/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:36.926497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875e7f9-1d75-4fcf-8cab-68e4699fb1c0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:37.034587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875e7f9-1d75-4fcf-8cab-68e4699fb1c0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:37.071834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:37.072574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:37.072851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:37.073120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:37.083609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:49:37.091439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:37.092678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:49:37.093753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 591/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:49:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:37.099208 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:37.197314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffa383-fb90-4f3d-9bc6-ce855153f58f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:37.233545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:37.234218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:37.234498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:37.234769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:37.235530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:37.235841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:37.237318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:37.245626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 590/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:37.254244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:37.295335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:37.373853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4d933-5f4e-4101-8611-362090807043 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:37.414909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:37.416213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:37.416575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:37.416918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] PATH_INFO: `/v3/users/89fbbb542fa24e4d997bb2155e3a0aee` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:37.434022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] RBAC: Authorizing `identity:get_user(user_id=89fbbb542fa24e4d997bb2155e3a0aee)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:37.435482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:37.437179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 592/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:37 2022] GET /identity/v3/users/89fbbb542fa24e4d997bb2155e3a0aee => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:37.443941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:37.500293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:37.555814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f2d06-e7be-421e-b61d-dab8090f322a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:37.592147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:37.592787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:37.593068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:37.593343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:37.662511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:37.663919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:37.709016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:37.766270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 591/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:37.776173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:37.875234 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:37.887053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f7aaf-a6d1-4cce-b083-6596282d2409 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:37.927503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ca34f-cf25-4ba0-bf81-2ebfa509da70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:37.928312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca34f-cf25-4ba0-bf81-2ebfa509da70 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:37.928592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca34f-cf25-4ba0-bf81-2ebfa509da70 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:37.928864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca34f-cf25-4ba0-bf81-2ebfa509da70 None admin] PATH_INFO: `/v3/projects/62dd191bb72343079076ad186badc68b` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:37.930183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ca34f-cf25-4ba0-bf81-2ebfa509da70 None admin] RBAC: Authorizing `identity:get_project(project_id=62dd191bb72343079076ad186badc68b)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:37.931643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ca34f-cf25-4ba0-bf81-2ebfa509da70 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:37.933393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 593/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:37 2022] GET /identity/v3/projects/62dd191bb72343079076ad186badc68b => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:37.943696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:38.055905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893857b-56d5-4afd-8ebf-819b6743b6b7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:38.092132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:38.092785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:38.093062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:38.093333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:38.094174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=62dd191bb72343079076ad186badc68b, user.id=89fbbb542fa24e4d997bb2155e3a0aee) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:38.094486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:38.095918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc791e-48cd-45a2-99bf-1d20f3a50bc9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:38.101564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 592/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:49:37 2022] GET /identity/v3/role_assignments?user.id=89fbbb542fa24e4d997bb2155e3a0aee&scope.project.id=62dd191bb72343079076ad186badc68b&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:40.154540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd699c7-c471-4a8b-884f-0f5f9c153341 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:40.155401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd699c7-c471-4a8b-884f-0f5f9c153341 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:40.155401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd699c7-c471-4a8b-884f-0f5f9c153341 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:40.157733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 594/1186] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:40.164075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:40.164500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:40.164670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:40.186891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:40.188105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:40.190022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:40.258599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 593/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:40.286404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd699c7-c471-4a8b-884f-0f5f9c153341 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:40.399756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd699c7-c471-4a8b-884f-0f5f9c153341 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:40.437171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:40.437890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:40.438167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:40.438441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:40.449119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:49:40.456568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:40.457606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:49:40.458974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 595/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:40.468345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:40.571517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3db0e1-d5cb-42d4-9a6f-6bcc5d2a9d2b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:40.608363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139514-694d-40b5-8973-01006c7be26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:40.609031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139514-694d-40b5-8973-01006c7be26a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:40.609311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139514-694d-40b5-8973-01006c7be26a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:40.609589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139514-694d-40b5-8973-01006c7be26a None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:40.610344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139514-694d-40b5-8973-01006c7be26a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:40.610651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139514-694d-40b5-8973-01006c7be26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:40.612054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139514-694d-40b5-8973-01006c7be26a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:40.617330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:40.621199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 594/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:40.628933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:40.656181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:40.741306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c92b1-6003-4723-8c54-296120451910 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:40.787153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:40.788407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:40.788692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:40.788968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] PATH_INFO: `/v3/users/89fbbb542fa24e4d997bb2155e3a0aee` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:40.790224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] RBAC: Authorizing `identity:get_user(user_id=89fbbb542fa24e4d997bb2155e3a0aee)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:40.791676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:40.793341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 596/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:40 2022] GET /identity/v3/users/89fbbb542fa24e4d997bb2155e3a0aee => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:40.799416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139514-694d-40b5-8973-01006c7be26a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:40.904418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139514-694d-40b5-8973-01006c7be26a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:40.938932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368e805-4cb5-4fba-b453-0142c7a61be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:40.939784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e805-4cb5-4fba-b453-0142c7a61be5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:40.940101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e805-4cb5-4fba-b453-0142c7a61be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:40.940375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e805-4cb5-4fba-b453-0142c7a61be5 None admin] PATH_INFO: `/v3/projects/62dd191bb72343079076ad186badc68b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:40.941683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e805-4cb5-4fba-b453-0142c7a61be5 None admin] RBAC: Authorizing `identity:get_project(project_id=62dd191bb72343079076ad186badc68b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:40.943114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e805-4cb5-4fba-b453-0142c7a61be5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:40.944967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 595/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:40 2022] GET /identity/v3/projects/62dd191bb72343079076ad186badc68b => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:40.951146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:41.053789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:41.066433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226d08f-3e86-439d-a66b-defbab6b8c8e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:41.102703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173265-b2cb-4214-8605-2ee7a932b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:41.104069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173265-b2cb-4214-8605-2ee7a932b024 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:41.104426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173265-b2cb-4214-8605-2ee7a932b024 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:41.104791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173265-b2cb-4214-8605-2ee7a932b024 None admin] PATH_INFO: `/v3/projects/62dd191bb72343079076ad186badc68b/users/89fbbb542fa24e4d997bb2155e3a0aee/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:41.107973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48173265-b2cb-4214-8605-2ee7a932b024 None admin] RBAC: Authorizing `identity:create_grant(project_id=62dd191bb72343079076ad186badc68b, user_id=89fbbb542fa24e4d997bb2155e3a0aee, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:41.110166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48173265-b2cb-4214-8605-2ee7a932b024 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:41.131542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 597/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:49:40 2022] PUT /identity/v3/projects/62dd191bb72343079076ad186badc68b/users/89fbbb542fa24e4d997bb2155e3a0aee/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:49:43.103274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9617c4b-e1fe-415a-98ef-c0cae8000056 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:43.104177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9617c4b-e1fe-415a-98ef-c0cae8000056 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:43.104177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9617c4b-e1fe-415a-98ef-c0cae8000056 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:43.106398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 596/1193] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:49:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:49:43.111713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:43.112152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:43.112273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:43.132148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:49:43.133315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:49:43.201210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 598/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:49:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:49:43.225131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9617c4b-e1fe-415a-98ef-c0cae8000056 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:43.310274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:43.353138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9617c4b-e1fe-415a-98ef-c0cae8000056 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:43.395472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:43.396227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:43.396607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:43.396809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:43.408785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:49:43.416373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:49:43.417453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:49:43.418629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 597/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:49:43.426848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:43.515378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:43.537327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfac9f6-d4a1-48dc-a4e2-1eab1e2591dd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:43.571652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:43.572337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:43.572613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:43.572881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:43.573627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:43.573930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:43.575309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:43.583350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 599/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:43.590432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:43.711636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4395a2-110c-41b1-9516-c6d4b64e68a3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:43.738673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:43.748010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:43.748876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:43.749149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:43.749412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] PATH_INFO: `/v3/users/89fbbb542fa24e4d997bb2155e3a0aee` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:43.750676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] RBAC: Authorizing `identity:get_user(user_id=89fbbb542fa24e4d997bb2155e3a0aee)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:43.752153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:43.753776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 598/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 09:49:43 2022] GET /identity/v3/users/89fbbb542fa24e4d997bb2155e3a0aee => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:43.763459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:43.876011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06946c0c-503c-4101-8909-07c308ba3807 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:43.910277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:43.910918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:43.911196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:43.911473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:43.921993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:43.986699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:43.988098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:44.103414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 600/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:49:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:49:44.110064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:44.221967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151ace9-2128-44bd-9a54-692ec629de0c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:44.266037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f4fff-031b-4dca-9c86-eaa23bac25c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:44.266831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4fff-031b-4dca-9c86-eaa23bac25c5 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:44.267111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4fff-031b-4dca-9c86-eaa23bac25c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:44.267382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4fff-031b-4dca-9c86-eaa23bac25c5 None admin] PATH_INFO: `/v3/projects/62dd191bb72343079076ad186badc68b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:44.268746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f4fff-031b-4dca-9c86-eaa23bac25c5 None admin] RBAC: Authorizing `identity:get_project(project_id=62dd191bb72343079076ad186badc68b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:44.270331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f4fff-031b-4dca-9c86-eaa23bac25c5 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:44.272185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 599/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 09:49:44 2022] GET /identity/v3/projects/62dd191bb72343079076ad186badc68b => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:44.278389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:49:44.321895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:44.395433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1070c-1c7f-4689-9899-156bea194327 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:49:44.430978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:49:44.431627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:49:44.431909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:49:44.432208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:49:44.433041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=62dd191bb72343079076ad186badc68b, user.id=89fbbb542fa24e4d997bb2155e3a0aee) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:49:44.433349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:49:44.434797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62895c-f714-4da2-a9b1-070b55114eba None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:49:44.440814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 601/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:49:44 2022] GET /identity/v3/role_assignments?user.id=89fbbb542fa24e4d997bb2155e3a0aee&scope.project.id=62dd191bb72343079076ad186badc68b&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:49:54.261001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=68 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:49:54.438017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=48 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:39.856583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01b675-3dde-4726-99c8-660de5a1939a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:39.857183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01b675-3dde-4726-99c8-660de5a1939a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:39.857183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01b675-3dde-4726-99c8-660de5a1939a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:39.858516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 600/1201] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:50:39.864167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:39.864640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:39.864737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:39.882642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:50:39.883837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:50:39.965615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 602/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:50:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:50:39.990349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01b675-3dde-4726-99c8-660de5a1939a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:40.124994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01b675-3dde-4726-99c8-660de5a1939a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:40.175572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:40.176457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:40.176726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:40.176992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:40.190297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:40.191756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:40.212232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:40.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:40.214908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:40.214908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:40.214908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:40.214908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:40.214908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 601/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:40.219681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:40.361010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce8faf-ee75-48a1-adba-86ce139998b6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:40.404675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:40.414568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:40.415318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:40.415585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:40.415854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:40.416703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:40.417008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:40.418690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e56d4-51ef-4346-b3de-3a30ff239643 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:40.439275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 603/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:40.439500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:40.509008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:40.575733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba318128-5e33-46a2-ad68-84eecbc5b3d0 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:40.620934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c2006-d2e9-484f-b847-15d5c6f82369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:40.621653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c2006-d2e9-484f-b847-15d5c6f82369 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:40.621905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c2006-d2e9-484f-b847-15d5c6f82369 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:40.622260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c2006-d2e9-484f-b847-15d5c6f82369 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:40.623063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c2006-d2e9-484f-b847-15d5c6f82369 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:40.624537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c2006-d2e9-484f-b847-15d5c6f82369 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:40.651010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 602/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 09:50:40 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:50:42.852349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7039967-d22e-4f9f-b101-ba4223e574ba None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:42.852934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7039967-d22e-4f9f-b101-ba4223e574ba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:42.852934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7039967-d22e-4f9f-b101-ba4223e574ba None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:42.854730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 604/1206] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:50:42.860766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:42.861264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:42.861496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:42.880792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:50:42.882073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:50:42.963966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 603/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:50:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:50:42.992908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7039967-d22e-4f9f-b101-ba4223e574ba None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:43.124909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7039967-d22e-4f9f-b101-ba4223e574ba None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:43.162029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:43.162845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:43.163240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:43.163556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:43.176186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:50:43.186745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:43.192519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:43.193753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 605/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:43.195270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:43.238805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:43.316184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fa4ff-2940-4e91-8363-daaac352680d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:43.359226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:43.360011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:43.360295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:43.360568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:43.361498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:43.361808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:43.363192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:43.372576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 604/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:43.378953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:43.447668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:43.504241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304586ca-0280-4c7b-b917-794232c81dac None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:43.540863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:43.541800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:43.542049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:43.542450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:43.554377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:43.555850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:43.579976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:43.581136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:43.582379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:43.582379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:43.582379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:43.582379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:43.588424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 606/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:43.596750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:43.711989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:43.715352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea980-1823-4d3a-86f3-af237dc632b9 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:43.758588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:43.759264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:43.759547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:43.759821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:43.760587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:43.760894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:43.762272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:43.777284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 605/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:43.783517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:43.896899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53393f76-0ce4-40e0-b089-efc290f9c002 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:43.898593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:43.937395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:43.938467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:43.938932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:43.939389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:43.947649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:43.949210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:50:43.967047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:43.968175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:43.969787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 607/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:50:43.974679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:44.091919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17d05c-42b8-446b-b476-177014194aa7 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:44.125393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:44.126080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:44.126411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:44.126688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:44.127535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:44.127912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:44.129790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:44.148708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 606/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 09:50:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:44.154669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:44.278194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddd6f5-29be-449d-a4c1-c5185342f712 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:44.314603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:44.315312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:44.315756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:44.315857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:44.350688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:44.395181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:44.396775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:44.503992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 608/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:50:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 09:50:44.510011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:44.533039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:44.630874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fee34-813c-4325-a4e6-32fb405e6131 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:44.670292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:44.671131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:44.671417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:44.671697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:44.683611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:44.685243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:44.705040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:44.706184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:44.707904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:44.707904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:44.707904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:44.707904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:44.710924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 607/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:44.713060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:44.778453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:44.826131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400fb2-0e50-4fbc-a11f-d42fd2de8484 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:44.863140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:44.863818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:44.864263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:44.864546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:44.865837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:44.866250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:44.867530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:44.880308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 609/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:44.885055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:44.969492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:45.008739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4762b-989f-4cdc-8269-8b1cf5a21fb4 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:45.051329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:45.052342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:45.052654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:45.052965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:45.061006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:45.063140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:50:45.080784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:45.081756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:45.082987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 608/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:45.089744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:45.215695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cc2b-b092-4d11-afb7-6094cad68546 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:45.241939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:45.252172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:45.252860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:45.253169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:45.253407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:45.254208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:45.254522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:45.256820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac260c-c991-418f-92c9-de27410b2bda None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:45.270378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 610/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:50:45.277627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:45.406787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f6a1-da42-42ed-b346-f9679979cbce None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:45.438860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:45.446486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:45.448342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:45.448342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:45.448342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:45.449189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=8641d4ad6c8a45f5b826bb43a78ee3da) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:45.449638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:45.451624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86d38a-9f24-451c-9d0b-382eabce11e7 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:45.464075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 609/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:50:45 2022] GET /identity/v3/role_assignments?user.id=8641d4ad6c8a45f5b826bb43a78ee3da&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:47.532180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb57aeb-1423-465a-b139-757d337b90c6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:47.532777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb57aeb-1423-465a-b139-757d337b90c6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:47.532777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb57aeb-1423-465a-b139-757d337b90c6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:47.534259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 611/1220] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:50:47.541649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:47.542060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:47.542164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:47.559368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:50:47.560859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:50:47.634902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 610/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:50:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:50:47.659626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb57aeb-1423-465a-b139-757d337b90c6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:47.775580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb57aeb-1423-465a-b139-757d337b90c6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:47.814825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:47.815657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:47.816095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:47.816416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:47.826830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:50:47.835030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:47.836163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:47.837191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 612/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:47.842370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:47.952172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e568-c5ce-4fb3-bf76-a62a6a12e175 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:47.981675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:47.994308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:47.995231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:47.995551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:47.995804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:47.996613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] RBAC: Adding query filter params (name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:47.997090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:47.998802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:48.008292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 611/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:50:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:48.015408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:48.066917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:48.131313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230cf14-bc62-46d3-9723-fd3349728be5 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:48.168668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:48.169509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:48.169841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:48.170254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:48.183102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:48.184636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:48.204378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:48.205484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:48.206483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:48.206483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:48.206483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:48.206483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:48.206483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 613/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:48.212800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:48.327845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604cc81-3740-421b-875e-d6decb34000d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:48.363371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:48.364347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:48.364632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:48.365081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:48.366035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:48.366494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:48.368066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:48.382088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 612/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:48.388183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:48.484714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:48.491974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab8a52-98f5-4c8e-bd0f-4b93c9e4679b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:48.526815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:48.527621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:48.527904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:48.528192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:48.536091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:48.537677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:50:48.551758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:48.553093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:48.554161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 614/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:48.559387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:48.609924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:48.666054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbd1b2-e3ee-40b8-9a5c-eb0aae58b304 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:48.703969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:48.704651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:48.704931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:48.705203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:48.706040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:48.706346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:48.708147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:48.727252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 613/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 09:50:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:48.733411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:48.841555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee68d0f-e92b-4eb0-926f-9704a1e17372 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:48.879567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:48.880632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:48.881057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:48.881306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:48.894340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:48.896321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:48.913308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:48.914393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:48.915408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:48.915408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:48.915408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:48.915408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:48.915408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 615/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:48.920946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:49.025416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598caa43-4822-4e2d-8231-a76fc886ad29 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:49.028075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:49.061224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:49.061917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:49.062319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:49.062684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:49.063530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:49.063982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:49.065452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:49.078136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 614/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:49.085166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:49.096205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:49.191711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789de133-65a6-4464-a192-9e2276b060a7 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:49.229605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:49.230664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:49.230973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:49.231323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:49.239459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:49.241200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:50:49.257923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:49.258954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:49.260059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 616/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:50:49.266767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:49.382566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c89fc-41c3-4aed-809d-bc9d24b3fbfb None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:49.420202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:49.420917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:49.421223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:49.421498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:49.422490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:49.422870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:49.424558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233133fc-253a-4af7-846c-2318ce78b11c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:49.436854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 615/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:49.443620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:49.517006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:49.556563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6307-d998-4691-971c-ba9188eaa201 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:49.601293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec205a-9c81-4bcd-b63a-cce476fa4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:49.602562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec205a-9c81-4bcd-b63a-cce476fa4026 None admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:49.602805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec205a-9c81-4bcd-b63a-cce476fa4026 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:49.603145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec205a-9c81-4bcd-b63a-cce476fa4026 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/8641d4ad6c8a45f5b826bb43a78ee3da/roles/68cf1edb4e1843bab4cc3da51fa364d6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:49.621968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec205a-9c81-4bcd-b63a-cce476fa4026 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=8641d4ad6c8a45f5b826bb43a78ee3da, role_id=68cf1edb4e1843bab4cc3da51fa364d6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:49.623612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec205a-9c81-4bcd-b63a-cce476fa4026 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:49.636574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 617/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:50:49 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/8641d4ad6c8a45f5b826bb43a78ee3da/roles/68cf1edb4e1843bab4cc3da51fa364d6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:50:51.944439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f325a-cce5-472c-abb8-f4f4775ebcfe None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:51.945096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f325a-cce5-472c-abb8-f4f4775ebcfe None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:51.945096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f325a-cce5-472c-abb8-f4f4775ebcfe None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:51.946671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 616/1233] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:50:51.952934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:51.953417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:51.953572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:51.973863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:50:51.975279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:50:52.044911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 618/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:50:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:50:52.069051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f325a-cce5-472c-abb8-f4f4775ebcfe None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:52.104189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:52.189304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f325a-cce5-472c-abb8-f4f4775ebcfe None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:52.227631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:52.228400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:52.228683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:52.228962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:52.240606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:50:52.248773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:52.250226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:52.251286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 617/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 09:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:52.256753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:52.306945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:52.360761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446954a-4345-4122-91e7-262e439e9e9d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:52.394658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:52.395329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:52.395622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:52.396002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:52.396822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] RBAC: Adding query filter params (name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:52.397095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:52.398465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:52.406755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 619/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 09:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:50:52.412895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:52.499641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:52.516406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e17d2-53f4-494f-bbbd-04a9e8b69f3a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:52.550584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:52.551404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:52.551685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:52.551991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:52.563411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:52.565071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:52.584191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:52.585313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:52.591120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:52.591120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:52.591120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:52.591120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:52.591120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 618/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:52.591522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:52.696438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff99d1f-3a4c-4469-9dd1-79672ca0f243 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:52.706191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:52.735936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:52.736623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:52.736938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:52.737271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:52.738077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:52.738407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:52.739992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:52.752819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 620/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:52.758618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:52.866248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a723-69f4-45af-8252-7f7c60abc0f2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:52.902927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:52.903808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:52.904188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:52.904462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:52.907536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:52.914125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:52.915784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:50:52.930557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:52.931773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:52.932896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 619/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:52.937741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:53.045710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a694a-3d04-4c2c-aede-854470e2da0e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:53.086457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:53.087141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:53.087490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:53.087839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:53.088894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:53.089211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:53.090753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:53.111323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 621/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 09:50:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:53.117758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:53.231974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae9b8-4431-41fe-b984-fe7b1cba795f None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:53.274501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:53.275167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:53.275551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:53.275822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:53.345460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:53.353257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:53.354724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:53.455505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 620/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:50:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:50:53.462131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:53.492328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:53.572862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf80d82-30a6-41d3-b286-8432a47ac797 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:53.610978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:53.611744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:53.612054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:53.612348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:53.623381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:53.624973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:53.642969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:53.643973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:53.644915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:53.644915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:53.644915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:53.644915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:53.644915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 622/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:53.650407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:53.752913 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:53.760420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628c47-0773-497c-9e71-87e649bb5cff None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:53.797792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:53.798467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:53.798743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:53.799024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:53.799757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:53.800096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:53.803260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:53.813048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 621/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:53.818979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:53.899183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:53.924050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a86f-2f39-4ba3-adc6-827d4e8c9197 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:53.962531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:53.963359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:53.963643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:53.963918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:53.970901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:53.972407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:50:53.987466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:53.988598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:53.989914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 623/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:53.995289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:54.117036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efa63f-7bd2-4747-b8bc-f36b1833e509 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:54.154241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:54.155023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:54.155156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:54.155427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:54.156317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:54.156625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:54.158306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f74c-d839-47de-856c-aba1594b29c1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:54.173473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 622/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:50:54.180038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:54.289623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966a56-ea7e-4f09-8e3f-bac7b8c1d92d None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:54.324452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:54.325106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:54.325385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:54.325656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:54.326481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] RBAC: Adding query filter params (role.id=68cf1edb4e1843bab4cc3da51fa364d6, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=8641d4ad6c8a45f5b826bb43a78ee3da) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:54.326789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:54.328279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccbdf5-f572-406f-aac7-f0380176cfd2 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:54.334518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 624/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:50:54 2022] GET /identity/v3/role_assignments?user.id=8641d4ad6c8a45f5b826bb43a78ee3da&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=68cf1edb4e1843bab4cc3da51fa364d6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:50:56.603340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d334e-494f-46d4-8adf-416184e21bbc None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:56.604003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d334e-494f-46d4-8adf-416184e21bbc None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:56.604003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d334e-494f-46d4-8adf-416184e21bbc None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:56.605999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 623/1247] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:50:56.612359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:56.612860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:56.613348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:56.619173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:56.632210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:50:56.633484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:50:56.714002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 625/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:50:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:50:56.739203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d334e-494f-46d4-8adf-416184e21bbc None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:56.759044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:56.854560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d334e-494f-46d4-8adf-416184e21bbc None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:56.890985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:56.891776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:56.892186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:56.892445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:56.903359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:50:56.911518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:56.912962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:56.914187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 624/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:56.919692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:57.029472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02c0d4-f4ae-4ef3-9090-711316d0085f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:57.063358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:57.069493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:57.070378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:57.070697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:57.070941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:57.071719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:57.072293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:57.073598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:57.085835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 626/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:57.092412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:57.181115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:57.205726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed7f2c-6a60-4156-bfb9-b343286b8437 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:57.243290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:57.244190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:57.244484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:57.244798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:57.255667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:57.257353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:57.278135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:57.279441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:57.285919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:57.285919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:57.285919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:57.285919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:57.285919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 625/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:57.286301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:57.395476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c0fd4-ddf6-4f0e-be04-0ac86a46888a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:57.439415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:57.440042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:57.440344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:57.440617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:57.441360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:57.441679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:57.443198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:57.455834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 627/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:57.462603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:57.581313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e70c3-f6c6-4e6c-bf84-0763438c09b6 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:57.614343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:57.624906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:57.625889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:57.626363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:57.626635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:57.636931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:57.638409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:50:57.653114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:57.654482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:50:57.655445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 626/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:57.663018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:57.732763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:57.787485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9356b4-f5af-4d39-be7f-b13c177d0e9e None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:57.831216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:57.832483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:57.832897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:57.833181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:57.834229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:57.834791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:57.837569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:57.857625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 628/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 09:50:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:57.865185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:57.980099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20519dbe-d438-4c3e-ae72-a22acb120ec2 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:58.020424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:58.021126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:58.021409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:58.021688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:58.086564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:58.102509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:58.104011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:58.219371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 627/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:50:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 09:50:58.225085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:58.335710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81321f0-99a8-486e-aefa-635a0d970545 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:58.350874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:58.374801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:58.375660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:58.375977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:58.376365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:58.387827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:58.389838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:50:58.407833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:58.408903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:58.409798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:50:58.409798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:50:58.409798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:50:58.409798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:58.409798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 629/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:58.415553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:58.512227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:58.524895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66402ea2-f834-459b-8fdc-17185d05192c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:58.566809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:58.567833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:58.568411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:58.568716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:58.569449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:58.569754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:58.571615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:58.586596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 628/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:50:58.594240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:58.701796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933413d-824f-4698-8a33-0e79264659fe None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:58.739105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:58.740113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:58.740465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:58.740643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:58.748548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:58.750037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:50:58.764622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:50:58.770563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:50:58.771589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 630/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:50:58.772234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:58.892113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb5400-27aa-4889-96f0-b6b568074b15 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:58.930360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:58.931027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:58.931302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:58.931571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:58.932393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:58.932694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:58.934130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68925333-536d-4fab-8279-789cf00ab35d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:58.948143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 629/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:50:58.954792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:50:58.984742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:50:59.068184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26221933-5fd0-4cf5-9ff4-9c3f8e93806a None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:50:59.111609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:50:59.112341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:50:59.112667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:50:59.113010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:50:59.113866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=8641d4ad6c8a45f5b826bb43a78ee3da) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:50:59.114189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:50:59.115847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7960a7-d48b-45e0-b5ef-68c02205b862 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:50:59.123005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 631/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:50:58 2022] GET /identity/v3/role_assignments?user.id=8641d4ad6c8a45f5b826bb43a78ee3da&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:01.230988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76d07-7966-41d0-982b-6be1b136d68c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:01.231558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76d07-7966-41d0-982b-6be1b136d68c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:01.231558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76d07-7966-41d0-982b-6be1b136d68c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:01.232965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 630/1261] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:51:01.238249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:01.238701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:01.238808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:01.256024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:51:01.257225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:51:01.328036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 632/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:51:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:51:01.351650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af76d07-7966-41d0-982b-6be1b136d68c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:01.364010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:01.459715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af76d07-7966-41d0-982b-6be1b136d68c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:01.497360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:01.498087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:01.498362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:01.498627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:01.510421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:51:01.517911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:01.518996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:01.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 631/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:01.525487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:01.544101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:01.634423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c95d0-cada-4c8f-b13f-c28604db4e6d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:01.672798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:01.673462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:01.673744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:01.674012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:01.674764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:01.675066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:01.676488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:01.684544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 633/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:01.694087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:01.766964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:01.804703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7284b-9975-4328-af85-8e121d7e3561 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:01.839248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:01.840055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:01.840330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:01.840602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:01.851434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:01.852914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:51:01.869780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:01.870808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:01.871835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:51:01.871835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:51:01.871835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:01.871835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:01.871835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 632/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:01.877571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:01.973106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:01.995357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b8412-f6e3-446c-b087-8e7894ad9831 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:02.032133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:02.032723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:02.033004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:02.033281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:02.034014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:02.034322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:02.035708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:02.048785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 634/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:02.056045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:02.169275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef4abd-973b-4149-bc28-316bda9895a1 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:02.188889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:02.204900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:02.205691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:02.205968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:02.206232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:02.214452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:02.215911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:51:02.230428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:02.232242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:02.234246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 633/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:51:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:02.244330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:02.353953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672f40e-c1e4-44c9-91de-2fc0ea70f151 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:02.389525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:02.390186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:02.390436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:02.390731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:02.391565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:02.391906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:02.393341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:02.402919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:02.412677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 635/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 09:51:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:02.419165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:02.530231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1b965-7763-4765-89eb-509a16a537d3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:02.565735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:02.566769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:02.567139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:02.567500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:02.585453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:02.586907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:51:02.604598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:02.605584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:02.610928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:51:02.610928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:51:02.610928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:02.610928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:02.610928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 634/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:02.611622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:02.728221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6951b4-bdb6-45e7-a97a-72859eb8274c None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:02.769643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:02.770332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:02.770693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:02.770971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:02.771703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:02.772043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:02.773425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:02.786208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 636/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:02.792273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:02.826422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:02.901586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bcb50-a41c-4346-9020-5439d58756e3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:02.936051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:02.936830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:02.937108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:02.937376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:02.944498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:02.945935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:51:02.960777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:02.961760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:02.962981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 635/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:02.969494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:03.035023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:03.085047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2272b8-6edc-49d2-8bf2-68cb13248db1 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:03.121047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:03.121708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:03.121990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:03.122262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:03.123059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:03.123366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:03.124858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0532e-6854-49d7-b8c4-67f2aac59555 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:03.136628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 637/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:51:03.143045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:03.242769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:03.252982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d4e2d-9a4b-4841-889f-179c6efa5f42 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:03.290225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89b4d8-3048-4b1e-a586-cbf938f26106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:03.291122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89b4d8-3048-4b1e-a586-cbf938f26106 None admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:03.291398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89b4d8-3048-4b1e-a586-cbf938f26106 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:03.292266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89b4d8-3048-4b1e-a586-cbf938f26106 None admin] PATH_INFO: `/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/8641d4ad6c8a45f5b826bb43a78ee3da/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:03.293663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89b4d8-3048-4b1e-a586-cbf938f26106 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3088f8a4154610998c2ab87ea883db, user_id=8641d4ad6c8a45f5b826bb43a78ee3da, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:03.295099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89b4d8-3048-4b1e-a586-cbf938f26106 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:03.310365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 636/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 09:51:03 2022] PUT /identity/v3/projects/9e3088f8a4154610998c2ab87ea883db/users/8641d4ad6c8a45f5b826bb43a78ee3da/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:51:05.292118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c143f5-bed4-4c00-9faf-d6ac4ccb20e4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:05.292572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c143f5-bed4-4c00-9faf-d6ac4ccb20e4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:05.292674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c143f5-bed4-4c00-9faf-d6ac4ccb20e4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:05.294066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 638/1274] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:51:05.300196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:05.300632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:05.300764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:05.324669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:51:05.325867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:51:05.399641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 637/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:51:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:51:05.427927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c143f5-bed4-4c00-9faf-d6ac4ccb20e4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:05.539280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c143f5-bed4-4c00-9faf-d6ac4ccb20e4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:05.557907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:05.587019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:05.587810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:05.588127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:05.588404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:05.599615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 09:51:05.607217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:05.608291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:05.609211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 639/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 09:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:05.616839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:05.710560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:05.741902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093db6e-c0e3-43a6-9fd2-255378a0092f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:05.777431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:05.778102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:05.778383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:05.778663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:05.779415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:05.779722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:05.781153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:05.790676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 638/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 09:51:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:05.796642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:05.904963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6710c0-81aa-4557-93cf-afd654f94af9 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:05.940244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:05.941037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:05.941415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:05.941702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:05.952820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:05.954279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:51:05.971902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:05.972994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:05.974461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:51:05.974461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:51:05.974461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:05.974461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:05.974461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 640/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 09:51:05.979718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:06.104303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c151efe-aef3-4fef-bb8d-7cf7cdcee7c3 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:06.140835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:06.141495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:06.141776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:06.142038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:06.142904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:06.143049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:06.144445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:06.156300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 639/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:06.164117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:06.197612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:06.271907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6f314-f7d0-4e72-b875-3f04251f8031 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:06.308651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:06.309455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:06.309731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:06.310000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:06.318428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:06.319895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 09:51:06.334416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:06.335466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:06.336809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 641/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:51:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:06.342129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:06.346609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:06.452345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25234cf5-63ae-49e4-9e12-01db2fc85cca None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:06.488218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:06.489052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:06.489211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:06.489511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:06.490343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:06.490646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:06.492093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:06.509446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 640/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 09:51:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:06.515541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:06.612154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:06.625235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12825e9-3d80-4594-b4fa-2be7f1893475 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:06.660156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:06.660889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:06.661182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:06.661472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:06.745474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:06.747679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:06.843709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:06.856570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 642/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 09:51:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 09:51:06.862616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:06.927690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:06.966477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7426f8-4457-4cb7-bfc0-feadd245099a None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:07.003417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:07.004156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:07.004430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:07.004720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:07.016067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:07.017502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 09:51:07.035066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:07.036267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:07.037522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 09:51:07.037522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 09:51:07.037522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 09:51:07.037522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:07.040073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 641/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:07.041961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:07.149189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5cedf-64f5-43c6-9b68-1e2fa7d6fdc6 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:07.183627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:07.184314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:07.184589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:07.184862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:07.185579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:07.185877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:07.187254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:07.198562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 643/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 09:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:07.204585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:07.318677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836355c-745a-4f04-a5fd-3dd5a130847c None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:07.334963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:07.357974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:07.358765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:07.359045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:07.359320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:07.366281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:07.367739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 09:51:07.381737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:07.382748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: ERROR keystone.server.flask.application Jun 30 09:51:07.383754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 642/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 09:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:07.391357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:07.445641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:07.498085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c667386-e64b-4136-9cd6-a84c1b33087b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:07.533650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:07.534302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:07.534587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:07.534863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:07.535665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:07.536151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:07.537595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a04a33-8514-481b-8ee7-c4c98fa795a4 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:07.549716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 644/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 09:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 09:51:07.559635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:07.667236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6c231-50fb-4b34-8c4f-db8bde54c20d None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:07.701923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:07.702559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:07.702793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:07.703101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:07.703953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] RBAC: Adding query filter params (role.id=d5c3d840a1cc493686f45bd92d196eac, scope.project.id=9e3088f8a4154610998c2ab87ea883db, user.id=8641d4ad6c8a45f5b826bb43a78ee3da) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:07.704274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:07.705699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3ab8b-d0aa-4ce4-9b71-da18f8d975b6 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:07.712043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 643/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 09:51:07 2022] GET /identity/v3/role_assignments?user.id=8641d4ad6c8a45f5b826bb43a78ee3da&scope.project.id=9e3088f8a4154610998c2ab87ea883db&role.id=d5c3d840a1cc493686f45bd92d196eac => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:09.720059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7952c1e-ab52-4014-844f-57fb1fce3bf7 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:09.720639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7952c1e-ab52-4014-844f-57fb1fce3bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:09.720639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7952c1e-ab52-4014-844f-57fb1fce3bf7 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:09.722766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 645/1288] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:51:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:51:09.728629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:09.729000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:09.729252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:09.748646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:51:09.749843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:51:09.788845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:09.829613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 644/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:51:09.852744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7952c1e-ab52-4014-844f-57fb1fce3bf7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:09.970352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7952c1e-ab52-4014-844f-57fb1fce3bf7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:09.982017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:10.013548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:10.014340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:10.014613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:10.014883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] PATH_INFO: `/v3/services/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:10.015545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:10.016940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:10.028916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:10.029971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:10.031190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 646/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:51:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:10.039502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:10.145089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0909c9-414a-4433-95bb-d5f39fe110c7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:10.190835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:10.191487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:10.191978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:10.192125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:10.192795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] RBAC: Adding query filter params (name=placement) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:10.193098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:10.194467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930d29-cbca-4a4d-9879-d09dfea5e1be None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:10.199490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 645/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 09:51:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:10.205691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:10.320023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb5d47-625b-4302-a048-1d2d71e513de None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:10.356296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:10.356943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:10.357222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:10.357489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] PATH_INFO: `/v3/services` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:10.358197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:10.358498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:10.359900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70198b-b7f9-4049-9a11-b177608f8d1a None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:10.364966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 647/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 09:51:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:12.450405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a679927-df2c-4beb-963d-0ff6740589e4 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:12.451353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a679927-df2c-4beb-963d-0ff6740589e4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:12.451353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a679927-df2c-4beb-963d-0ff6740589e4 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:12.453581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 646/1293] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:51:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:51:12.462109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cd002-b3f6-49eb-86d5-84d1c55cb310 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:12.462535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cd002-b3f6-49eb-86d5-84d1c55cb310 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:12.462678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cd002-b3f6-49eb-86d5-84d1c55cb310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:12.490786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1b9cd002-b3f6-49eb-86d5-84d1c55cb310 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:51:12.491991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1b9cd002-b3f6-49eb-86d5-84d1c55cb310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:51:12.522385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:12.563147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 648/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:51:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:51:12.586777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a679927-df2c-4beb-963d-0ff6740589e4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:12.621974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:12.699065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a679927-df2c-4beb-963d-0ff6740589e4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:12.735165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409ea33-303b-4c43-9675-a964e2452ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:12.735805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ea33-303b-4c43-9675-a964e2452ad8 None admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:12.736116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ea33-303b-4c43-9675-a964e2452ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:12.736391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ea33-303b-4c43-9675-a964e2452ad8 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:12.737129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409ea33-303b-4c43-9675-a964e2452ad8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:12.738499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409ea33-303b-4c43-9675-a964e2452ad8 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:12.754139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 647/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 09:51:12 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:51:14.771287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e30655-3b68-402a-b365-a404cd2bd380 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:14.771914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e30655-3b68-402a-b365-a404cd2bd380 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:14.771914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e30655-3b68-402a-b365-a404cd2bd380 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:14.773283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 649/1296] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:51:14.778586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:14.779063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:14.779164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:14.804245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:51:14.805463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:51:14.881942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 648/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:51:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 09:51:14.906234 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e30655-3b68-402a-b365-a404cd2bd380 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:15.028713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e30655-3b68-402a-b365-a404cd2bd380 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:15.078740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:15.079650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:15.080028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:15.080315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:15.081054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:15.082935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:15.098106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:15.099347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:15.100315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 650/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:51:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:15.107053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:15.128502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:15.214144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf627a47-e2e5-449c-bacf-e5919826c561 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:15.250466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:15.251122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:15.251401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:15.251670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:15.252415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:15.252723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:15.254102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:15.259439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 649/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 09:51:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:15.266276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:15.304642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:15.376409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf61e5-6779-41a3-9da2-3c26d6f0813b None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:15.410846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:15.411497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:15.411805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:15.412299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:15.412887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] RBAC: Adding query filter params (interface=public, service_id=d49c6464ef014669a721f5fdd26b1548, region_id=RegionOne) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:15.413196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:15.414589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6004baf-a8a9-4954-a578-e5ff780e5213 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:15.419736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 651/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 09:51:15 2022] GET /identity/v3/endpoints?service_id=d49c6464ef014669a721f5fdd26b1548&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:15.425868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:15.514682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:15.534100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25271ef7-d9e7-4aaf-8c3f-842c03e8e149 None admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:15.571159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225eee7-79bf-47b6-be18-354e2b9de396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:15.571741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225eee7-79bf-47b6-be18-354e2b9de396 None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:15.572045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225eee7-79bf-47b6-be18-354e2b9de396 None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:15.572318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225eee7-79bf-47b6-be18-354e2b9de396 None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:15.572998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225eee7-79bf-47b6-be18-354e2b9de396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:15.574357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225eee7-79bf-47b6-be18-354e2b9de396 None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:15.581210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 650/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 09:51:15 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:51:17.560008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b293e-eb46-4666-8d63-692da596732f None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:17.560548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b293e-eb46-4666-8d63-692da596732f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:17.560548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b293e-eb46-4666-8d63-692da596732f None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:17.561853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 652/1302] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:51:17.568719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12479e-3150-450f-893b-07ab61044cec None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:17.569151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12479e-3150-450f-893b-07ab61044cec None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:17.569260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12479e-3150-450f-893b-07ab61044cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:17.587823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4d12479e-3150-450f-893b-07ab61044cec None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:51:17.589014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4d12479e-3150-450f-893b-07ab61044cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:51:17.658052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 651/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:51:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 09:51:17.688953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b293e-eb46-4666-8d63-692da596732f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:17.799705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:17.811997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b293e-eb46-4666-8d63-692da596732f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:17.849911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:17.850728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:17.851005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:17.851308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:17.852006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:17.853409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 09:51:17.866519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 09:51:17.867643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application with Lock( Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self._enter() Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application return self.creator() Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application created_value = creator( Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: ERROR keystone.server.flask.application Jun 30 09:51:17.869147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 653/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 09:51:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 09:51:17.877827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12479e-3150-450f-893b-07ab61044cec None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:17.956081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:17.981256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12479e-3150-450f-893b-07ab61044cec None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:18.015466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:18.016140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:18.016418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:18.016686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] PATH_INFO: `/v3/services` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:18.017383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:51:18.017684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:18.019048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2249ec2-bc2b-4507-9e15-d909cc9ce51d None admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:18.024329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 652/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 09:51:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:51:18.030823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:51:18.139534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ce63d-fdfc-4e7f-b88f-8a4c6fcbf151 None admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:51:18.175915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a47e0-7065-4935-a76f-dc8609f277ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:51:18.176587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a47e0-7065-4935-a76f-dc8609f277ba None admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:51:18.176866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a47e0-7065-4935-a76f-dc8609f277ba None admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:51:18.177140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a47e0-7065-4935-a76f-dc8609f277ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:51:18.178238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a47e0-7065-4935-a76f-dc8609f277ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:51:18.179732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a47e0-7065-4935-a76f-dc8609f277ba None admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:51:18.202673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 654/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 09:51:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:51:28.022786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=30 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:51:28.197959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:21.214328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4689d-3e20-4881-98ce-bd79625c7a71 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:21.214910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4689d-3e20-4881-98ce-bd79625c7a71 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:21.214910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4689d-3e20-4881-98ce-bd79625c7a71 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:21.221746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fefd01-6c52-4b23-acce-fb4508e9d5d8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:21.222180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fefd01-6c52-4b23-acce-fb4508e9d5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:21.222267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fefd01-6c52-4b23-acce-fb4508e9d5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:21.225869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 653/1307] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:21.341594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b7fefd01-6c52-4b23-acce-fb4508e9d5d8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:21.342966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b7fefd01-6c52-4b23-acce-fb4508e9d5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:21.456677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 655/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:21.503430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff10bf-b408-42d8-8298-67e94cfcbc3f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:21.503725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff10bf-b408-42d8-8298-67e94cfcbc3f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:21.504047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff10bf-b408-42d8-8298-67e94cfcbc3f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:21.505831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 654/1309] 10.222.0.16 () {54 vars in 964 bytes} [Thu Jun 30 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:21.511990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89122be-3888-45e3-8745-cfed951e4dc2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:21.512305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89122be-3888-45e3-8745-cfed951e4dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:21.512500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89122be-3888-45e3-8745-cfed951e4dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:21.640506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a89122be-3888-45e3-8745-cfed951e4dc2 None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:21.641730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a89122be-3888-45e3-8745-cfed951e4dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:21.667155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:21.756910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 656/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Jun 30 09:52:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:21.766119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff10bf-b408-42d8-8298-67e94cfcbc3f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:21.768659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a2ff10bf-b408-42d8-8298-67e94cfcbc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:21.889586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff10bf-b408-42d8-8298-67e94cfcbc3f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:21.964541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:21.965197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:21.965483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:21.965755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:21.967417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:22.023559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:22.082327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:22.083896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfbe96-2699-44c4-8d02-016328db8187 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:22.103604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 655/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:24.433640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26a9d-5f4a-4e01-9562-88b1a15d6516 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:24.434194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26a9d-5f4a-4e01-9562-88b1a15d6516 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:24.434194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f26a9d-5f4a-4e01-9562-88b1a15d6516 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:24.435574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 657/1312] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:24.440923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c172-9246-4c8a-a7ab-2e5e142d76c0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:24.441379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c172-9246-4c8a-a7ab-2e5e142d76c0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:24.441470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c172-9246-4c8a-a7ab-2e5e142d76c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:24.521673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2676c172-9246-4c8a-a7ab-2e5e142d76c0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:24.522880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2676c172-9246-4c8a-a7ab-2e5e142d76c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:24.621932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 656/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:24.674192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929521ea-17c9-4492-ac3b-331b89a3ba46 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:24.674838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929521ea-17c9-4492-ac3b-331b89a3ba46 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:24.675193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929521ea-17c9-4492-ac3b-331b89a3ba46 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:24.676732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 658/1314] 10.222.0.16 () {54 vars in 964 bytes} [Thu Jun 30 09:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:24.682951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89475-1d2d-44eb-8d1c-d3544ce038c8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:24.683238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89475-1d2d-44eb-8d1c-d3544ce038c8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:24.683507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89475-1d2d-44eb-8d1c-d3544ce038c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:24.694235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:24.765138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a1d89475-1d2d-44eb-8d1c-d3544ce038c8 None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:24.766399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a1d89475-1d2d-44eb-8d1c-d3544ce038c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:24.853645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 657/1315] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Jun 30 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:24.863375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929521ea-17c9-4492-ac3b-331b89a3ba46 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:24.865305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-929521ea-17c9-4492-ac3b-331b89a3ba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:24.975157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929521ea-17c9-4492-ac3b-331b89a3ba46 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:25.050162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:25.053371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:25.054028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:25.054314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:25.054597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:25.056778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:25.171323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:25.172820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decf708-5ee4-4146-b3f5-ea92a50189a6 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:25.191560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 659/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:27.729167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea125f7-adcc-44c7-a1de-344562ef2d92 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:27.729707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea125f7-adcc-44c7-a1de-344562ef2d92 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:27.729707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea125f7-adcc-44c7-a1de-344562ef2d92 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:27.731121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 658/1317] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:52:27.736542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd407da-497a-46d1-b6da-21b2ffbf9829 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:27.736981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd407da-497a-46d1-b6da-21b2ffbf9829 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:27.737074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd407da-497a-46d1-b6da-21b2ffbf9829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:27.822777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-abd407da-497a-46d1-b6da-21b2ffbf9829 None None] MFA Rules not processed for user `0e717e6887484c4c914aefb3e73d3210`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:27.824011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-abd407da-497a-46d1-b6da-21b2ffbf9829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:27.885558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:27.937192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 660/1318] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:27.989953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea125f7-adcc-44c7-a1de-344562ef2d92 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:27.991952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cea125f7-adcc-44c7-a1de-344562ef2d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:28.119736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea125f7-adcc-44c7-a1de-344562ef2d92 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:28.123290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:28.241424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:28.241983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:28.242264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:28.242532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:28.244222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:28.373001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:28.375385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4403-d06c-4f1c-a02c-1a421fb40845 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:28.402249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 659/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:31.072870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e67c41-1049-4f59-ab84-6e0b0af80012 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:31.073421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e67c41-1049-4f59-ab84-6e0b0af80012 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:31.073421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e67c41-1049-4f59-ab84-6e0b0af80012 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:31.074856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 661/1320] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:31.081464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808ce4-d05d-49fe-9333-0abe630441b4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:31.082184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808ce4-d05d-49fe-9333-0abe630441b4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:31.082341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808ce4-d05d-49fe-9333-0abe630441b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:31.123726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:31.154671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4d808ce4-d05d-49fe-9333-0abe630441b4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:31.155916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4d808ce4-d05d-49fe-9333-0abe630441b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:31.257610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 660/1321] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:31.298853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e67c41-1049-4f59-ab84-6e0b0af80012 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:31.300499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-31e67c41-1049-4f59-ab84-6e0b0af80012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:31.406568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e67c41-1049-4f59-ab84-6e0b0af80012 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:31.501953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:31.502596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:31.502872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:31.503148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:31.505416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:31.515348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:31.638434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:31.639908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadebf54-a1cf-49b0-a31d-eb026f080482 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:31.658923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 662/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:34.514341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6981d8-bd55-4f4a-89e1-cd35d6703bce None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:34.514911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6981d8-bd55-4f4a-89e1-cd35d6703bce None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:34.514911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6981d8-bd55-4f4a-89e1-cd35d6703bce None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:34.516358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 661/1323] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:34.522221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628003dd-2e9f-47e9-8599-9e3e06377f76 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:34.522666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628003dd-2e9f-47e9-8599-9e3e06377f76 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:34.522781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628003dd-2e9f-47e9-8599-9e3e06377f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:34.599647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-628003dd-2e9f-47e9-8599-9e3e06377f76 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:34.600933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-628003dd-2e9f-47e9-8599-9e3e06377f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:34.668448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:34.720032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 663/1324] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:34.756435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6981d8-bd55-4f4a-89e1-cd35d6703bce None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:34.757995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0d6981d8-bd55-4f4a-89e1-cd35d6703bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:34.883058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6981d8-bd55-4f4a-89e1-cd35d6703bce None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:34.896030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:34.958915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:34.959494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:34.959769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:34.960066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:34.961699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:35.074408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:35.075856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8442d58-c86e-48ed-8a3d-4300021663d5 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:35.094458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 662/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:37.153405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478ab01-e787-4b55-b47f-021301e14ca8 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:37.153932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478ab01-e787-4b55-b47f-021301e14ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:37.153932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478ab01-e787-4b55-b47f-021301e14ca8 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:37.155336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 664/1326] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:37.160887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc358-2ec6-43a2-9521-2fad6cf49450 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:37.161450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc358-2ec6-43a2-9521-2fad6cf49450 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:37.161684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc358-2ec6-43a2-9521-2fad6cf49450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:37.209138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:37.227907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-950bc358-2ec6-43a2-9521-2fad6cf49450 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:37.229134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-950bc358-2ec6-43a2-9521-2fad6cf49450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:37.331560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 663/1327] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:37.367977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478ab01-e787-4b55-b47f-021301e14ca8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:37.369624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5478ab01-e787-4b55-b47f-021301e14ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:37.485467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478ab01-e787-4b55-b47f-021301e14ca8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:37.563593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:37.564286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:37.564568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:37.564843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:37.566577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:37.594076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:37.691680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:37.693164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c427ce-ba01-456b-9041-cf04c6c379de service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:37.712572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 665/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:39.788297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e4b9a-b83f-4cb8-853c-cbb956219023 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:39.788948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e4b9a-b83f-4cb8-853c-cbb956219023 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:39.788948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e4b9a-b83f-4cb8-853c-cbb956219023 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:39.790269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 664/1329] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:39.796316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8883fd2-6263-41e4-a944-f118b331c14e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:39.796929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8883fd2-6263-41e4-a944-f118b331c14e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:39.796929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8883fd2-6263-41e4-a944-f118b331c14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:39.866244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c8883fd2-6263-41e4-a944-f118b331c14e None None] MFA Rules not processed for user `0e717e6887484c4c914aefb3e73d3210`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:39.867416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c8883fd2-6263-41e4-a944-f118b331c14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:39.934728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:39.969585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 666/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:40.007770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e4b9a-b83f-4cb8-853c-cbb956219023 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:40.008754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-552e4b9a-b83f-4cb8-853c-cbb956219023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:40.116364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e4b9a-b83f-4cb8-853c-cbb956219023 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:40.136135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:40.212944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:40.213614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:40.213896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:40.214170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:40.215861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:40.333957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:40.335407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9aa13-abff-44fa-9181-b2cd2758f4f6 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:40.354928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 665/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:42.799851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ad3bc-d5b9-4857-8fcb-2cfa57e8b942 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:42.800821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ad3bc-d5b9-4857-8fcb-2cfa57e8b942 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:42.800821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ad3bc-d5b9-4857-8fcb-2cfa57e8b942 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:42.803509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 667/1332] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:42.809956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3922-e7a0-4ffb-a149-e708b193e722 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:42.810693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3922-e7a0-4ffb-a149-e708b193e722 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:42.810873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3922-e7a0-4ffb-a149-e708b193e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:42.882547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:42.885910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0e9c3922-e7a0-4ffb-a149-e708b193e722 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:42.887586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0e9c3922-e7a0-4ffb-a149-e708b193e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:42.983902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 666/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:43.022102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ad3bc-d5b9-4857-8fcb-2cfa57e8b942 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:43.023640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-491ad3bc-d5b9-4857-8fcb-2cfa57e8b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:43.135458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ad3bc-d5b9-4857-8fcb-2cfa57e8b942 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:43.218178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:43.218836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:43.219116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:43.219388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:43.221045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:43.247538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:43.339903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:43.341388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce634c-bf24-4327-bc4c-b1257a781c8c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:43.360040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 668/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:45.426696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d81d3-cd53-4844-8cc0-1a8f7dade9ff None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:45.427224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d81d3-cd53-4844-8cc0-1a8f7dade9ff None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:45.427224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d81d3-cd53-4844-8cc0-1a8f7dade9ff None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:45.429118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 667/1335] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:45.437823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d6076-8189-4dfa-a2a4-d668f5a806d3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:45.438219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d6076-8189-4dfa-a2a4-d668f5a806d3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:45.438422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d6076-8189-4dfa-a2a4-d668f5a806d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:45.512017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-084d6076-8189-4dfa-a2a4-d668f5a806d3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:45.513244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-084d6076-8189-4dfa-a2a4-d668f5a806d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:45.591657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:45.614542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 669/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:45.660275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d81d3-cd53-4844-8cc0-1a8f7dade9ff None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:45.662521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-673d81d3-cd53-4844-8cc0-1a8f7dade9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:45.784193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d81d3-cd53-4844-8cc0-1a8f7dade9ff None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:45.813085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:45.868318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:45.868966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:45.869248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:45.869519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:45.871165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:46.001298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:46.002757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7559b-4108-487d-b6ad-ad12b72995af service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:46.021974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 668/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:48.510822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66e2a-cdf4-439e-9080-8dcb590b47f5 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:48.511768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66e2a-cdf4-439e-9080-8dcb590b47f5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:48.511768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66e2a-cdf4-439e-9080-8dcb590b47f5 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:48.513782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 670/1338] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:48.520579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d429ee-76fb-4ac5-83cc-23e93f609670 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:48.521334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d429ee-76fb-4ac5-83cc-23e93f609670 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:48.521551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d429ee-76fb-4ac5-83cc-23e93f609670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:48.595018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b4d429ee-76fb-4ac5-83cc-23e93f609670 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:48.596317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b4d429ee-76fb-4ac5-83cc-23e93f609670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:48.606430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:48.701327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 669/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:48.738003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b66e2a-cdf4-439e-9080-8dcb590b47f5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:48.740655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-25b66e2a-cdf4-439e-9080-8dcb590b47f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:48.865341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b66e2a-cdf4-439e-9080-8dcb590b47f5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:48.940782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:48.941444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:48.941720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:48.941987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:48.943623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:48.981751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:49.062408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:49.063898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a0497-c231-4fee-946a-244a1ce0cd4d service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:49.084180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 671/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:53.104590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd04369-2b47-4f55-b57f-1c2535a02a61 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:53.105089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd04369-2b47-4f55-b57f-1c2535a02a61 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:53.105190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd04369-2b47-4f55-b57f-1c2535a02a61 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:53.106536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 670/1341] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:52:53.112070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008224-6c14-42f9-9565-ee823a568553 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:53.112508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008224-6c14-42f9-9565-ee823a568553 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:53.112681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008224-6c14-42f9-9565-ee823a568553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:53.185891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6b008224-6c14-42f9-9565-ee823a568553 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:53.187087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6b008224-6c14-42f9-9565-ee823a568553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:53.287006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:53.301771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 672/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:52:53.340787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd04369-2b47-4f55-b57f-1c2535a02a61 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:53.343268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fdd04369-2b47-4f55-b57f-1c2535a02a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:53.442658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd04369-2b47-4f55-b57f-1c2535a02a61 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:53.446529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e94dd8-b34d-488a-a969-7a17d06204ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:53.447182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e94dd8-b34d-488a-a969-7a17d06204ae service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:53.447463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e94dd8-b34d-488a-a969-7a17d06204ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:53.447743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e94dd8-b34d-488a-a969-7a17d06204ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:53.457562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e94dd8-b34d-488a-a969-7a17d06204ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:53.459016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e94dd8-b34d-488a-a969-7a17d06204ae service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:53.468347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 671/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:52:56.111908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6ce7-2f67-493e-b7a7-a9f050e2fdb0 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:56.112471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6ce7-2f67-493e-b7a7-a9f050e2fdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:56.112471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6ce7-2f67-493e-b7a7-a9f050e2fdb0 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:56.113954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 673/1344] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:52:56.119566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b235-e3b1-4fa4-86e6-24701145ead9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:56.120039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b235-e3b1-4fa4-86e6-24701145ead9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:56.120275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b235-e3b1-4fa4-86e6-24701145ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:56.149126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d602b235-e3b1-4fa4-86e6-24701145ead9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:52:56.150322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d602b235-e3b1-4fa4-86e6-24701145ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:56.181203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:52:56.235897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 672/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:52:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:52:56.275054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6ce7-2f67-493e-b7a7-a9f050e2fdb0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:52:56.277118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d57f6ce7-2f67-493e-b7a7-a9f050e2fdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:52:56.296778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6ce7-2f67-493e-b7a7-a9f050e2fdb0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:52:56.300709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134854b0-9bcd-4d85-a649-bb49d44d70c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:52:56.301461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134854b0-9bcd-4d85-a649-bb49d44d70c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:52:56.301825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134854b0-9bcd-4d85-a649-bb49d44d70c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:52:56.302084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134854b0-9bcd-4d85-a649-bb49d44d70c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:52:56.312096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134854b0-9bcd-4d85-a649-bb49d44d70c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:52:56.313568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134854b0-9bcd-4d85-a649-bb49d44d70c8 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:52:56.324384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 674/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:52:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:00.654813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaeba8-fea1-4da7-81ac-40bccb544e1c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:00.655395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaeba8-fea1-4da7-81ac-40bccb544e1c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:00.655395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaeba8-fea1-4da7-81ac-40bccb544e1c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:00.656777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 673/1347] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:00.662458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795513-2d90-4386-bb72-6c3c357836ec None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:00.662897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795513-2d90-4386-bb72-6c3c357836ec None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:00.662991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795513-2d90-4386-bb72-6c3c357836ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:00.681717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-57795513-2d90-4386-bb72-6c3c357836ec None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:00.682939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-57795513-2d90-4386-bb72-6c3c357836ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:00.691939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 675/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:00.731449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaeba8-fea1-4da7-81ac-40bccb544e1c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:00.742817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaeba8-fea1-4da7-81ac-40bccb544e1c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:00.746533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266114e7-88e5-4117-a090-316f58fc1036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:00.747168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266114e7-88e5-4117-a090-316f58fc1036 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:00.747452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266114e7-88e5-4117-a090-316f58fc1036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:00.747726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266114e7-88e5-4117-a090-316f58fc1036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:00.757160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266114e7-88e5-4117-a090-316f58fc1036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:00.758668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266114e7-88e5-4117-a090-316f58fc1036 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:00.768844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 674/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:05.769340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e468-a8c3-4878-9a6e-1695f50fc0e5 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:05.769847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e468-a8c3-4878-9a6e-1695f50fc0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:05.769847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e468-a8c3-4878-9a6e-1695f50fc0e5 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:05.771373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 676/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 09:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:05.777063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88bfa-be39-4208-b43f-b4176111bcd0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:05.777507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88bfa-be39-4208-b43f-b4176111bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:05.777601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88bfa-be39-4208-b43f-b4176111bcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:05.885942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-11b88bfa-be39-4208-b43f-b4176111bcd0 None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:05.887264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-11b88bfa-be39-4208-b43f-b4176111bcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:05.969710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 675/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 09:53:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:10.015330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5861cf-f2b4-45a7-b27e-6c553973bfba None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:10.016330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5861cf-f2b4-45a7-b27e-6c553973bfba None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:10.016330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5861cf-f2b4-45a7-b27e-6c553973bfba None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:10.017484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 677/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 09:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:10.022573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69021e58-e1b5-4358-9223-191b36d0fc20 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:10.023044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69021e58-e1b5-4358-9223-191b36d0fc20 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:10.023152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69021e58-e1b5-4358-9223-191b36d0fc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:10.043893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-69021e58-e1b5-4358-9223-191b36d0fc20 None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:10.045101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-69021e58-e1b5-4358-9223-191b36d0fc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:10.054488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 676/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 09:53:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:53:10.676644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=13 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:53:11.613407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb003-a601-4cfc-89c5-b369d7d179cb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:11.614087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb003-a601-4cfc-89c5-b369d7d179cb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:11.614087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb003-a601-4cfc-89c5-b369d7d179cb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:11.615541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 678/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 09:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:11.620466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482dab3-a7ae-4373-b3fb-7e2b44a9caea None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:11.621221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482dab3-a7ae-4373-b3fb-7e2b44a9caea None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:11.621525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482dab3-a7ae-4373-b3fb-7e2b44a9caea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:11.640360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3482dab3-a7ae-4373-b3fb-7e2b44a9caea None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:11.641551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3482dab3-a7ae-4373-b3fb-7e2b44a9caea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:11.650757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 677/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 09:53:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:53:15.272359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc8964-07a7-4194-8b0c-af9a98340860 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:15.272855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc8964-07a7-4194-8b0c-af9a98340860 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:15.272855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc8964-07a7-4194-8b0c-af9a98340860 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:15.274335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 679/1356] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:15.281980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a950dd1-5a24-42ac-90eb-eaee49bc3dec None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:15.282823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a950dd1-5a24-42ac-90eb-eaee49bc3dec None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:15.283101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a950dd1-5a24-42ac-90eb-eaee49bc3dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:15.310048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3a950dd1-5a24-42ac-90eb-eaee49bc3dec None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:15.311224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3a950dd1-5a24-42ac-90eb-eaee49bc3dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:15.319514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 678/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:15.376845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8c7c4-3bde-4fef-b51d-1c7b2b510401 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:15.377670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8c7c4-3bde-4fef-b51d-1c7b2b510401 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:15.378104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8c7c4-3bde-4fef-b51d-1c7b2b510401 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:15.380388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 680/1358] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 09:53:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:15.391822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a95a-9a8c-4b95-8c6f-f140e5468fd0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:15.391822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a95a-9a8c-4b95-8c6f-f140e5468fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:15.391822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a95a-9a8c-4b95-8c6f-f140e5468fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:15.436568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:53:15.444523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c0d2a95a-9a8c-4b95-8c6f-f140e5468fd0 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:15.445778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c0d2a95a-9a8c-4b95-8c6f-f140e5468fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:15.532984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 679/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 09:53:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:53:15.540995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8c7c4-3bde-4fef-b51d-1c7b2b510401 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:15.556774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8c7c4-3bde-4fef-b51d-1c7b2b510401 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:15.562765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdcfc2-7825-4aeb-8992-ab4cbf72d4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:15.563418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdcfc2-7825-4aeb-8992-ab4cbf72d4e5 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:15.563695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdcfc2-7825-4aeb-8992-ab4cbf72d4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:15.564002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdcfc2-7825-4aeb-8992-ab4cbf72d4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:15.573717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdcfc2-7825-4aeb-8992-ab4cbf72d4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:15.575334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdcfc2-7825-4aeb-8992-ab4cbf72d4e5 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:15.586420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 681/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:17.818756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9f5f-f1c4-41ff-837a-917d5010fd4d None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:17.819379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9f5f-f1c4-41ff-837a-917d5010fd4d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:17.819379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9f5f-f1c4-41ff-837a-917d5010fd4d None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:17.820921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 680/1361] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:17.826045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca506f5-c774-4b2e-8ec8-6b21da60fda3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:17.826494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca506f5-c774-4b2e-8ec8-6b21da60fda3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:17.826610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca506f5-c774-4b2e-8ec8-6b21da60fda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:17.846698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3ca506f5-c774-4b2e-8ec8-6b21da60fda3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:17.848397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3ca506f5-c774-4b2e-8ec8-6b21da60fda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:17.858020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 682/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:17.901212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9f5f-f1c4-41ff-837a-917d5010fd4d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:17.917524 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9f5f-f1c4-41ff-837a-917d5010fd4d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:17.924418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e52e2-a35c-40d2-a445-a5867dabb0cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:17.925293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e52e2-a35c-40d2-a445-a5867dabb0cb service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:17.925754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e52e2-a35c-40d2-a445-a5867dabb0cb service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:17.926182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e52e2-a35c-40d2-a445-a5867dabb0cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:17.935705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e52e2-a35c-40d2-a445-a5867dabb0cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:17.937834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e52e2-a35c-40d2-a445-a5867dabb0cb service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:17.949736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 681/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:20.366674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830217f5-cd5e-489f-be4f-f94728445f01 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:20.367242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830217f5-cd5e-489f-be4f-f94728445f01 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:20.367242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830217f5-cd5e-489f-be4f-f94728445f01 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:20.368679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 683/1364] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:20.374333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c4e3d-d494-426b-9afa-caefd374e2c0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:20.374907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c4e3d-d494-426b-9afa-caefd374e2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:20.374907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c4e3d-d494-426b-9afa-caefd374e2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:20.393745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ba7c4e3d-d494-426b-9afa-caefd374e2c0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:20.395037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ba7c4e3d-d494-426b-9afa-caefd374e2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:20.402920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 682/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:20.448414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190b0a-9dc7-4ca2-90a2-a0eb7478c701 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:20.448699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190b0a-9dc7-4ca2-90a2-a0eb7478c701 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:20.448970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190b0a-9dc7-4ca2-90a2-a0eb7478c701 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:20.450590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 684/1366] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 09:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:20.456535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816781ce-dc61-4e2f-93e1-9fc80942f93f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:20.456819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816781ce-dc61-4e2f-93e1-9fc80942f93f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:20.457089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816781ce-dc61-4e2f-93e1-9fc80942f93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:20.489862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-816781ce-dc61-4e2f-93e1-9fc80942f93f None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:20.491214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-816781ce-dc61-4e2f-93e1-9fc80942f93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:20.501781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 683/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 09:53:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:20.510049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190b0a-9dc7-4ca2-90a2-a0eb7478c701 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:20.526119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190b0a-9dc7-4ca2-90a2-a0eb7478c701 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:20.531835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94275b2-0a04-4ac6-9bf7-42d2482df0fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:20.532511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94275b2-0a04-4ac6-9bf7-42d2482df0fa service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:20.532795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94275b2-0a04-4ac6-9bf7-42d2482df0fa service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:20.533071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94275b2-0a04-4ac6-9bf7-42d2482df0fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:20.541585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94275b2-0a04-4ac6-9bf7-42d2482df0fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:20.543186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94275b2-0a04-4ac6-9bf7-42d2482df0fa service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:20.554715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 685/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:22.676712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc450814-4b89-479d-a5bb-71cac894cb42 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:22.677281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc450814-4b89-479d-a5bb-71cac894cb42 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:22.677281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc450814-4b89-479d-a5bb-71cac894cb42 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:22.679092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 684/1369] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:22.684879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe6b6-8d90-4fe7-9199-86b9e390fca1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:22.685380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe6b6-8d90-4fe7-9199-86b9e390fca1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:22.685380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe6b6-8d90-4fe7-9199-86b9e390fca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:22.704384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-27cfe6b6-8d90-4fe7-9199-86b9e390fca1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:22.705645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-27cfe6b6-8d90-4fe7-9199-86b9e390fca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:22.713931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 686/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:22.756902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc450814-4b89-479d-a5bb-71cac894cb42 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:22.773266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc450814-4b89-479d-a5bb-71cac894cb42 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:22.781483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a37ca-8419-4cdd-9a06-5fdc5c7321a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:22.782392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a37ca-8419-4cdd-9a06-5fdc5c7321a1 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:22.782914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a37ca-8419-4cdd-9a06-5fdc5c7321a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:22.783513 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a37ca-8419-4cdd-9a06-5fdc5c7321a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:22.792619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a37ca-8419-4cdd-9a06-5fdc5c7321a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:22.794563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a37ca-8419-4cdd-9a06-5fdc5c7321a1 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:22.805014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 685/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:23.475811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612040c-f349-42e2-b11d-42e42d1daab1 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:23.476072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612040c-f349-42e2-b11d-42e42d1daab1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:23.476350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612040c-f349-42e2-b11d-42e42d1daab1 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:23.477947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 687/1372] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:23.483373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e06199-2a28-4f19-b0c0-ae8425a6f730 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:23.483903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e06199-2a28-4f19-b0c0-ae8425a6f730 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:23.484290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e06199-2a28-4f19-b0c0-ae8425a6f730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:23.505517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e0e06199-2a28-4f19-b0c0-ae8425a6f730 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:23.507408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e0e06199-2a28-4f19-b0c0-ae8425a6f730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:23.518265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 686/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:23.871176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b70249-19e7-493d-bf2f-6e8f111f06f8 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:23.872001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b70249-19e7-493d-bf2f-6e8f111f06f8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:23.872252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b70249-19e7-493d-bf2f-6e8f111f06f8 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:23.873796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 688/1374] 10.222.0.16 () {54 vars in 963 bytes} [Thu Jun 30 09:53:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:23.879313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ef3e-3837-4b77-9c7b-eeee46eb3a92 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:23.879785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ef3e-3837-4b77-9c7b-eeee46eb3a92 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:23.880064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ef3e-3837-4b77-9c7b-eeee46eb3a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:23.947628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-64e0ef3e-3837-4b77-9c7b-eeee46eb3a92 None None] MFA Rules not processed for user `15924f58d53145ae83b3b4cfbbb02ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:23.949163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-64e0ef3e-3837-4b77-9c7b-eeee46eb3a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:24.018980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 687/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Jun 30 09:53:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:53:24.023603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b70249-19e7-493d-bf2f-6e8f111f06f8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:24.035078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b70249-19e7-493d-bf2f-6e8f111f06f8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:24.039122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a974-aff5-4229-be4f-fed9ed61dbc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:24.040068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a974-aff5-4229-be4f-fed9ed61dbc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:24.040570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a974-aff5-4229-be4f-fed9ed61dbc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:24.041074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a974-aff5-4229-be4f-fed9ed61dbc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:24.051857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a974-aff5-4229-be4f-fed9ed61dbc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:24.053458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a974-aff5-4229-be4f-fed9ed61dbc4 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:24.065413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 689/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:24.848933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dc82a-4a22-4409-86cb-f80775a30ec3 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:24.849194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dc82a-4a22-4409-86cb-f80775a30ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:24.849504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dc82a-4a22-4409-86cb-f80775a30ec3 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:24.851126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 688/1377] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:24.856503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da835af8-330b-4134-8458-9b49f864ce93 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:24.856790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da835af8-330b-4134-8458-9b49f864ce93 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:24.857060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da835af8-330b-4134-8458-9b49f864ce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:24.874684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-da835af8-330b-4134-8458-9b49f864ce93 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:24.875862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-da835af8-330b-4134-8458-9b49f864ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:24.883843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 690/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:24.927907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dc82a-4a22-4409-86cb-f80775a30ec3 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:24.942840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dc82a-4a22-4409-86cb-f80775a30ec3 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:24.949099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2629120-5699-4a30-a43b-e688fcc48969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:24.949741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2629120-5699-4a30-a43b-e688fcc48969 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:24.950040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2629120-5699-4a30-a43b-e688fcc48969 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:24.950308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2629120-5699-4a30-a43b-e688fcc48969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:24.959735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2629120-5699-4a30-a43b-e688fcc48969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:24.961353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2629120-5699-4a30-a43b-e688fcc48969 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:24.965329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:53:24.982474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 689/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:27.037012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7450-a60b-44bb-92c7-1c4d63e5a5b4 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:27.038532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7450-a60b-44bb-92c7-1c4d63e5a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:27.039011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7450-a60b-44bb-92c7-1c4d63e5a5b4 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:27.041411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 691/1380] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:27.047138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65637ae5-4aba-4bfb-a2a6-8a5049b56163 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:27.047997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65637ae5-4aba-4bfb-a2a6-8a5049b56163 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:27.048473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65637ae5-4aba-4bfb-a2a6-8a5049b56163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:27.071029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-65637ae5-4aba-4bfb-a2a6-8a5049b56163 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:27.072288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-65637ae5-4aba-4bfb-a2a6-8a5049b56163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:27.081295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 690/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:27.132510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b7450-a60b-44bb-92c7-1c4d63e5a5b4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:27.147312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b7450-a60b-44bb-92c7-1c4d63e5a5b4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:27.152901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e376-13f9-48c0-9d54-ec2facfc4126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:27.153549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e376-13f9-48c0-9d54-ec2facfc4126 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:27.153847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e376-13f9-48c0-9d54-ec2facfc4126 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:27.154118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e376-13f9-48c0-9d54-ec2facfc4126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:27.162690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e376-13f9-48c0-9d54-ec2facfc4126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:27.164359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e376-13f9-48c0-9d54-ec2facfc4126 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:27.174181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 692/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:29.276879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3ee9-b546-4761-a08c-10b65cf3dd0f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:29.277763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3ee9-b546-4761-a08c-10b65cf3dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:29.278189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda3ee9-b546-4761-a08c-10b65cf3dd0f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:29.279999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 691/1383] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:29.287717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34609e9-8736-4c66-ab4f-e3941db2b3dd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:29.289411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34609e9-8736-4c66-ab4f-e3941db2b3dd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:29.289917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34609e9-8736-4c66-ab4f-e3941db2b3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:29.315140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a34609e9-8736-4c66-ab4f-e3941db2b3dd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:29.317053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a34609e9-8736-4c66-ab4f-e3941db2b3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:29.328120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 693/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:29.376641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3ee9-b546-4761-a08c-10b65cf3dd0f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:29.391332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda3ee9-b546-4761-a08c-10b65cf3dd0f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:29.397407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf5cc1-2424-4ccb-97d4-f10cb399fc5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:29.398061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5cc1-2424-4ccb-97d4-f10cb399fc5e service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:29.398340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5cc1-2424-4ccb-97d4-f10cb399fc5e service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:29.398612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5cc1-2424-4ccb-97d4-f10cb399fc5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:29.407498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf5cc1-2424-4ccb-97d4-f10cb399fc5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:29.409391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf5cc1-2424-4ccb-97d4-f10cb399fc5e service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:29.419670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 692/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:31.484969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea3396-7c16-411e-b231-3931dd2fbe84 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:31.485586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea3396-7c16-411e-b231-3931dd2fbe84 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:31.485586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea3396-7c16-411e-b231-3931dd2fbe84 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:31.487925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 694/1386] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:31.494485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070939a-4cd8-4f01-b2c1-b73e90a5fd5e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:31.494485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070939a-4cd8-4f01-b2c1-b73e90a5fd5e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:31.494485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070939a-4cd8-4f01-b2c1-b73e90a5fd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:31.528588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f070939a-4cd8-4f01-b2c1-b73e90a5fd5e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:31.532098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f070939a-4cd8-4f01-b2c1-b73e90a5fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:31.547994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 693/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:31.593698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea3396-7c16-411e-b231-3931dd2fbe84 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:31.611031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea3396-7c16-411e-b231-3931dd2fbe84 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:31.617923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44072c6-e8c8-4d78-b898-f004ae1e7e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:31.618570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44072c6-e8c8-4d78-b898-f004ae1e7e2e service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:31.618850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44072c6-e8c8-4d78-b898-f004ae1e7e2e service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:31.619122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44072c6-e8c8-4d78-b898-f004ae1e7e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:31.628182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44072c6-e8c8-4d78-b898-f004ae1e7e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:31.629780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44072c6-e8c8-4d78-b898-f004ae1e7e2e service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:31.640657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 695/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:33.777556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a95b-f60c-4cbc-a97c-c9a19a70e0fe None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:33.778113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a95b-f60c-4cbc-a97c-c9a19a70e0fe None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:33.778226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a95b-f60c-4cbc-a97c-c9a19a70e0fe None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:33.780094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 694/1389] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:33.787287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c7ede-b235-498a-aa12-cf1840332167 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:33.787913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c7ede-b235-498a-aa12-cf1840332167 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:33.787913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c7ede-b235-498a-aa12-cf1840332167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:33.806482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0f5c7ede-b235-498a-aa12-cf1840332167 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:33.807669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0f5c7ede-b235-498a-aa12-cf1840332167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:33.815995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 696/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:33.860592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747a95b-f60c-4cbc-a97c-c9a19a70e0fe None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:33.877947 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747a95b-f60c-4cbc-a97c-c9a19a70e0fe None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:33.883339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22b92a-c74e-4ed7-9a70-1346ab6c142e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:33.884012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22b92a-c74e-4ed7-9a70-1346ab6c142e service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:33.884359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22b92a-c74e-4ed7-9a70-1346ab6c142e service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:33.884550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22b92a-c74e-4ed7-9a70-1346ab6c142e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:33.893464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22b92a-c74e-4ed7-9a70-1346ab6c142e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:33.895079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22b92a-c74e-4ed7-9a70-1346ab6c142e service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:33.904749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 695/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:36.375564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211efc12-1475-48dd-9892-f0efb260cc94 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:36.376183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211efc12-1475-48dd-9892-f0efb260cc94 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:36.376371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211efc12-1475-48dd-9892-f0efb260cc94 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:36.378441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 697/1392] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:36.383976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512c266-d2ae-4b73-b726-8e152f5dbbd0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:36.384576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512c266-d2ae-4b73-b726-8e152f5dbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:36.384576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512c266-d2ae-4b73-b726-8e152f5dbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:36.406709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f512c266-d2ae-4b73-b726-8e152f5dbbd0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:36.408006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f512c266-d2ae-4b73-b726-8e152f5dbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:36.416805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 696/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:36.462503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211efc12-1475-48dd-9892-f0efb260cc94 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:36.477403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211efc12-1475-48dd-9892-f0efb260cc94 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:36.482730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6f555-5506-41ca-846e-e55973c04b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:36.483377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6f555-5506-41ca-846e-e55973c04b47 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:36.483722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6f555-5506-41ca-846e-e55973c04b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:36.484005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6f555-5506-41ca-846e-e55973c04b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:36.492473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6f555-5506-41ca-846e-e55973c04b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:36.494079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6f555-5506-41ca-846e-e55973c04b47 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:36.504316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 698/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:38.543048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdae363-9035-4daa-9095-88cad334ca3f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:38.543576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdae363-9035-4daa-9095-88cad334ca3f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:38.543576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdae363-9035-4daa-9095-88cad334ca3f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:38.544939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 697/1395] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:38.551004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0c80b-3bd7-4d04-8ba9-67a5eba49bb2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:38.551439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0c80b-3bd7-4d04-8ba9-67a5eba49bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:38.551747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0c80b-3bd7-4d04-8ba9-67a5eba49bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:38.572550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e8f0c80b-3bd7-4d04-8ba9-67a5eba49bb2 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:38.573767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e8f0c80b-3bd7-4d04-8ba9-67a5eba49bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:38.582793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 699/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:38.626279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdae363-9035-4daa-9095-88cad334ca3f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:38.641919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdae363-9035-4daa-9095-88cad334ca3f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:38.648096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1ebb6-1617-4508-8f0e-98ab8c774c9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:38.648742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1ebb6-1617-4508-8f0e-98ab8c774c9e service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:38.649023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1ebb6-1617-4508-8f0e-98ab8c774c9e service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:38.649296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1ebb6-1617-4508-8f0e-98ab8c774c9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:38.658216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1ebb6-1617-4508-8f0e-98ab8c774c9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:38.659827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1ebb6-1617-4508-8f0e-98ab8c774c9e service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:38.669684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 698/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:39.379839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4d0ce-448d-4426-be86-f5e50fa9efac None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:39.380345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4d0ce-448d-4426-be86-f5e50fa9efac None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:39.380772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4d0ce-448d-4426-be86-f5e50fa9efac None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:39.382978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 700/1398] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:39.391760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7d5de-63de-45f0-b9e6-7969c32b8d0e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:39.392489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7d5de-63de-45f0-b9e6-7969c32b8d0e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:39.392873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7d5de-63de-45f0-b9e6-7969c32b8d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:39.411100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8ce7d5de-63de-45f0-b9e6-7969c32b8d0e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:39.412317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8ce7d5de-63de-45f0-b9e6-7969c32b8d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:39.420902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 699/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:53:39.482240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecd63c-643f-4124-84be-cbe4a9495304 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:39.482532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecd63c-643f-4124-84be-cbe4a9495304 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:39.482808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecd63c-643f-4124-84be-cbe4a9495304 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:39.484245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 701/1400] 10.222.0.16 () {54 vars in 962 bytes} [Thu Jun 30 09:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:39.490679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16368d9-852e-412f-8c73-404041c24a22 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:39.491103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16368d9-852e-412f-8c73-404041c24a22 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:39.491369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16368d9-852e-412f-8c73-404041c24a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:39.541921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b16368d9-852e-412f-8c73-404041c24a22 None None] MFA Rules not processed for user `0a98b63a4ad14d7399938ffb9c6b018a`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:39.543131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b16368d9-852e-412f-8c73-404041c24a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:39.604956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 700/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Jun 30 09:53:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:39.612898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecd63c-643f-4124-84be-cbe4a9495304 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:39.624786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecd63c-643f-4124-84be-cbe4a9495304 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:39.628418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2186cc-3f64-4bef-a80a-4898f85af1fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:39.629106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2186cc-3f64-4bef-a80a-4898f85af1fe service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:39.629341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2186cc-3f64-4bef-a80a-4898f85af1fe service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:39.629638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2186cc-3f64-4bef-a80a-4898f85af1fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:39.638664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2186cc-3f64-4bef-a80a-4898f85af1fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:39.640155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2186cc-3f64-4bef-a80a-4898f85af1fe service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:39.649543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 702/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:40.384789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78183-2a2c-4b1d-87a8-8c82791b2535 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.385051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78183-2a2c-4b1d-87a8-8c82791b2535 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.385472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78183-2a2c-4b1d-87a8-8c82791b2535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.433563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-93f78183-2a2c-4b1d-87a8-8c82791b2535 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:40.434791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-93f78183-2a2c-4b1d-87a8-8c82791b2535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:40.477276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:53:40.501696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 701/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 09:53:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:40.546033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264a525-b32a-4c26-9ef9-df141d6892db None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.546383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264a525-b32a-4c26-9ef9-df141d6892db None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.546721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264a525-b32a-4c26-9ef9-df141d6892db None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.548200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 703/1404] 10.222.0.16 () {54 vars in 942 bytes} [Thu Jun 30 09:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:40.555093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f4a7-07ef-4ceb-a4f6-4cfac79b81f9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.555598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f4a7-07ef-4ceb-a4f6-4cfac79b81f9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.555700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f4a7-07ef-4ceb-a4f6-4cfac79b81f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.605752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4fa2f4a7-07ef-4ceb-a4f6-4cfac79b81f9 None None] MFA Rules not processed for user `64bf7840b5fd4b3b9dd2b8e18b4495b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:40.606981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4fa2f4a7-07ef-4ceb-a4f6-4cfac79b81f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:40.682826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 702/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Thu Jun 30 09:53:40 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:40.692398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264a525-b32a-4c26-9ef9-df141d6892db None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:40.699760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=61 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:53:40.708250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264a525-b32a-4c26-9ef9-df141d6892db None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:40.713805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dba45-885b-481b-a6d4-8fab36bca546 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:40.714530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dba45-885b-481b-a6d4-8fab36bca546 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.714873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dba45-885b-481b-a6d4-8fab36bca546 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.715204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dba45-885b-481b-a6d4-8fab36bca546 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.723701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dba45-885b-481b-a6d4-8fab36bca546 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:40.725386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dba45-885b-481b-a6d4-8fab36bca546 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:40.733431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 704/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 09:53:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 09:53:40.756894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496819-ecee-4b44-b829-96b9cae6880d None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.757192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496819-ecee-4b44-b829-96b9cae6880d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.757476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496819-ecee-4b44-b829-96b9cae6880d None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.758884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 703/1407] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:53:40.764335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98afcb9-7252-48c3-a5f2-26749e5ed926 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.764774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98afcb9-7252-48c3-a5f2-26749e5ed926 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.764972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98afcb9-7252-48c3-a5f2-26749e5ed926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.783631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f98afcb9-7252-48c3-a5f2-26749e5ed926 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:53:40.784886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f98afcb9-7252-48c3-a5f2-26749e5ed926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:53:40.793615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 705/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:53:40.840498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496819-ecee-4b44-b829-96b9cae6880d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:53:40.855754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496819-ecee-4b44-b829-96b9cae6880d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:53:40.864406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d2954-f4c9-4884-afc9-05c9e4e97c5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:53:40.865155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2954-f4c9-4884-afc9-05c9e4e97c5d service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:53:40.865493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2954-f4c9-4884-afc9-05c9e4e97c5d service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:53:40.865832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2954-f4c9-4884-afc9-05c9e4e97c5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:53:40.875075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d2954-f4c9-4884-afc9-05c9e4e97c5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:53:40.876852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d2954-f4c9-4884-afc9-05c9e4e97c5d service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:53:40.886884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 704/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:53:50.778093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=8 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:53:50.882180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=59 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:54:02.706081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d068007-9838-40b0-9620-2d7a19e24a6c None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:02.706709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d068007-9838-40b0-9620-2d7a19e24a6c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:02.706709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d068007-9838-40b0-9620-2d7a19e24a6c None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:02.708044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 706/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Jun 30 09:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:02.713339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5364-5c2c-4306-b4cf-d7213221eac6 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:02.713784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5364-5c2c-4306-b4cf-d7213221eac6 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:02.713884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5364-5c2c-4306-b4cf-d7213221eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:02.732873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f05d5364-5c2c-4306-b4cf-d7213221eac6 None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:02.734175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f05d5364-5c2c-4306-b4cf-d7213221eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:02.743355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 705/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jun 30 09:54:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:02.757535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c5169-20c8-4a2b-a57c-cfdee433b1d2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:02.757813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c5169-20c8-4a2b-a57c-cfdee433b1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:02.758098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c5169-20c8-4a2b-a57c-cfdee433b1d2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:02.759463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 707/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu Jun 30 09:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:02.764782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9b4cc-22d8-4329-aac5-44185b4cbe7d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:02.765169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9b4cc-22d8-4329-aac5-44185b4cbe7d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:02.765525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9b4cc-22d8-4329-aac5-44185b4cbe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:02.790642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-7de9b4cc-22d8-4329-aac5-44185b4cbe7d None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:02.791834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7de9b4cc-22d8-4329-aac5-44185b4cbe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:02.802469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 706/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jun 30 09:54:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:05.294367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a09816-702d-4455-bbce-0d5ec4ca6298 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:05.294796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a09816-702d-4455-bbce-0d5ec4ca6298 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:05.294934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a09816-702d-4455-bbce-0d5ec4ca6298 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:05.297164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:05.305554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf03bdf-a2ff-472a-9f96-d64e8d77dc8a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:05.305929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf03bdf-a2ff-472a-9f96-d64e8d77dc8a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:05.306098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf03bdf-a2ff-472a-9f96-d64e8d77dc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:05.328343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4bf03bdf-a2ff-472a-9f96-d64e8d77dc8a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:05.329557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4bf03bdf-a2ff-472a-9f96-d64e8d77dc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:05.339707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:05.390765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a09816-702d-4455-bbce-0d5ec4ca6298 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:05.406970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a09816-702d-4455-bbce-0d5ec4ca6298 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:05.413236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e12ca1-4c91-4c80-9359-ff84cf1328af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:05.413878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12ca1-4c91-4c80-9359-ff84cf1328af service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:05.414162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12ca1-4c91-4c80-9359-ff84cf1328af service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:05.414449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e12ca1-4c91-4c80-9359-ff84cf1328af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:05.422787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e12ca1-4c91-4c80-9359-ff84cf1328af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:05.424965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e12ca1-4c91-4c80-9359-ff84cf1328af service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:05.434355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 709/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:06.360505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835138-8eaf-4980-9bfb-68b37d7524d3 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:06.361457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835138-8eaf-4980-9bfb-68b37d7524d3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:06.361623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835138-8eaf-4980-9bfb-68b37d7524d3 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:06.363037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:06.368912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e67e-72f3-4295-b99a-26b4ebe265bd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:06.369588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e67e-72f3-4295-b99a-26b4ebe265bd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:06.369696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e67e-72f3-4295-b99a-26b4ebe265bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:06.397230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d627e67e-72f3-4295-b99a-26b4ebe265bd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:06.398422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d627e67e-72f3-4295-b99a-26b4ebe265bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:06.408294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:06.558455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835138-8eaf-4980-9bfb-68b37d7524d3 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:06.575511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835138-8eaf-4980-9bfb-68b37d7524d3 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:06.585843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4644f24-f24d-49b9-a168-13471e1937a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:06.586792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4644f24-f24d-49b9-a168-13471e1937a4 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:06.587216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4644f24-f24d-49b9-a168-13471e1937a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:06.587596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4644f24-f24d-49b9-a168-13471e1937a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:06.599971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4644f24-f24d-49b9-a168-13471e1937a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:06.601586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4644f24-f24d-49b9-a168-13471e1937a4 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:06.611396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 709/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:07.498260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650888-d6a6-4139-b946-49b92dd773c6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:07.498817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650888-d6a6-4139-b946-49b92dd773c6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:07.498817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650888-d6a6-4139-b946-49b92dd773c6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:07.500268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:07.505972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad4a9a-3709-4cb0-ae1f-a8c0fd7987ca None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:07.506413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad4a9a-3709-4cb0-ae1f-a8c0fd7987ca None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:07.506525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad4a9a-3709-4cb0-ae1f-a8c0fd7987ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:07.526855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-25ad4a9a-3709-4cb0-ae1f-a8c0fd7987ca None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:07.528112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-25ad4a9a-3709-4cb0-ae1f-a8c0fd7987ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:07.538073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:07.688883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f650888-d6a6-4139-b946-49b92dd773c6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:07.706719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f650888-d6a6-4139-b946-49b92dd773c6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:07.714255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4131d7-4028-409b-ae84-a6e79f9df365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:07.714900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4131d7-4028-409b-ae84-a6e79f9df365 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:07.715178 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4131d7-4028-409b-ae84-a6e79f9df365 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:07.715465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4131d7-4028-409b-ae84-a6e79f9df365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:07.724894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4131d7-4028-409b-ae84-a6e79f9df365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:07.726496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4131d7-4028-409b-ae84-a6e79f9df365 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:07.736046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 712/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:08.600671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdcdbb-35b9-4ed2-aad2-0b98e8e84c79 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:08.601558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdcdbb-35b9-4ed2-aad2-0b98e8e84c79 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:08.601558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdcdbb-35b9-4ed2-aad2-0b98e8e84c79 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:08.603685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:08.611984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a875-8b9e-46e7-89df-6cb1f2ecdf29 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:08.612408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a875-8b9e-46e7-89df-6cb1f2ecdf29 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:08.612540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194a875-8b9e-46e7-89df-6cb1f2ecdf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:08.630913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b194a875-8b9e-46e7-89df-6cb1f2ecdf29 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:08.632174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b194a875-8b9e-46e7-89df-6cb1f2ecdf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:08.640434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:08.800859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdcdbb-35b9-4ed2-aad2-0b98e8e84c79 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:08.822558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdcdbb-35b9-4ed2-aad2-0b98e8e84c79 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:08.829661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1663f42-be6d-41ef-9ca3-262502fec3c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:08.830293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1663f42-be6d-41ef-9ca3-262502fec3c3 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:08.830708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1663f42-be6d-41ef-9ca3-262502fec3c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:08.830915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1663f42-be6d-41ef-9ca3-262502fec3c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:08.839541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1663f42-be6d-41ef-9ca3-262502fec3c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:08.841161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1663f42-be6d-41ef-9ca3-262502fec3c3 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:08.851320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 712/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:09.718933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b28d-50ad-416c-8d67-0c87ada8703b None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:09.719856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b28d-50ad-416c-8d67-0c87ada8703b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:09.719856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b28d-50ad-416c-8d67-0c87ada8703b None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:09.721885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:09.726896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d554c-8c4a-4c47-8031-a34355c375dc None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:09.727338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d554c-8c4a-4c47-8031-a34355c375dc None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:09.727432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d554c-8c4a-4c47-8031-a34355c375dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:09.746859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c38d554c-8c4a-4c47-8031-a34355c375dc None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:09.748101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c38d554c-8c4a-4c47-8031-a34355c375dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:09.757803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:09.914193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b28d-50ad-416c-8d67-0c87ada8703b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:09.934631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b28d-50ad-416c-8d67-0c87ada8703b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:09.941210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c3500-eaba-4af4-8bf2-2e809d206e9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:09.941852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c3500-eaba-4af4-8bf2-2e809d206e9e service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:09.942131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c3500-eaba-4af4-8bf2-2e809d206e9e service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:09.942398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c3500-eaba-4af4-8bf2-2e809d206e9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:09.951404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c3500-eaba-4af4-8bf2-2e809d206e9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:09.953448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c3500-eaba-4af4-8bf2-2e809d206e9e service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:09.963468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 715/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:10.767299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b306dbc-4eae-4136-bab0-8381c213b8ed None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:10.767927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b306dbc-4eae-4136-bab0-8381c213b8ed None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:10.767927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b306dbc-4eae-4136-bab0-8381c213b8ed None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:10.769381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:10.774631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0dd6-c68f-442a-b5fb-42a63637ee46 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:10.775113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0dd6-c68f-442a-b5fb-42a63637ee46 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:10.775239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0dd6-c68f-442a-b5fb-42a63637ee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:10.796537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e9ae0dd6-c68f-442a-b5fb-42a63637ee46 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:10.797736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e9ae0dd6-c68f-442a-b5fb-42a63637ee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:10.808451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:10.960376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b306dbc-4eae-4136-bab0-8381c213b8ed None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:10.981651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b306dbc-4eae-4136-bab0-8381c213b8ed None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:10.991260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5912b-b2e1-4077-8f77-f685c688aaee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:10.992153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5912b-b2e1-4077-8f77-f685c688aaee service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:10.992515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5912b-b2e1-4077-8f77-f685c688aaee service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:10.992857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5912b-b2e1-4077-8f77-f685c688aaee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:11.004686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5912b-b2e1-4077-8f77-f685c688aaee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:11.006262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5912b-b2e1-4077-8f77-f685c688aaee service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:11.015632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 715/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:11.822792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83de4-dd8d-48ff-94ac-c9ba506ff493 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:11.823695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83de4-dd8d-48ff-94ac-c9ba506ff493 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:11.823695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83de4-dd8d-48ff-94ac-c9ba506ff493 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:11.825920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:11.834455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57c89-d1a8-4442-8d9e-f3f768507594 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:11.834901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57c89-d1a8-4442-8d9e-f3f768507594 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:11.835082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57c89-d1a8-4442-8d9e-f3f768507594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:11.858476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-95a57c89-d1a8-4442-8d9e-f3f768507594 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:11.859687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-95a57c89-d1a8-4442-8d9e-f3f768507594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:11.868424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:12.018180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f83de4-dd8d-48ff-94ac-c9ba506ff493 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:12.039435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f83de4-dd8d-48ff-94ac-c9ba506ff493 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:12.045954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566634c4-b287-47c4-a190-2b3d5f191ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:12.046594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566634c4-b287-47c4-a190-2b3d5f191ec3 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:12.046875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566634c4-b287-47c4-a190-2b3d5f191ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:12.047149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566634c4-b287-47c4-a190-2b3d5f191ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:12.055960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566634c4-b287-47c4-a190-2b3d5f191ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:12.057584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566634c4-b287-47c4-a190-2b3d5f191ec3 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:12.067076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 718/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:12.881801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c87bb-4313-448e-bbcf-b55f0ed1ecb0 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:12.882693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c87bb-4313-448e-bbcf-b55f0ed1ecb0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:12.882693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c87bb-4313-448e-bbcf-b55f0ed1ecb0 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:12.884900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:12.893518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3bbb4-0c9a-456a-8169-d78e71605fc6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:12.893949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3bbb4-0c9a-456a-8169-d78e71605fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:12.894086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3bbb4-0c9a-456a-8169-d78e71605fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:12.913013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-52d3bbb4-0c9a-456a-8169-d78e71605fc6 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:12.914202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-52d3bbb4-0c9a-456a-8169-d78e71605fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:12.923063 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:13.074731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c87bb-4313-448e-bbcf-b55f0ed1ecb0 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:13.090848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c87bb-4313-448e-bbcf-b55f0ed1ecb0 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:13.096688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefad4c-89fa-48c2-9d27-b9ea450fe21f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:13.097321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefad4c-89fa-48c2-9d27-b9ea450fe21f service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:13.097599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefad4c-89fa-48c2-9d27-b9ea450fe21f service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:13.097863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefad4c-89fa-48c2-9d27-b9ea450fe21f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:13.106748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefad4c-89fa-48c2-9d27-b9ea450fe21f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:13.108382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefad4c-89fa-48c2-9d27-b9ea450fe21f service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:13.118122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 718/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:13.915343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dd057-e9c0-4df3-bc7e-7fcb9f90a4a6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:13.916452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dd057-e9c0-4df3-bc7e-7fcb9f90a4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:13.916452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dd057-e9c0-4df3-bc7e-7fcb9f90a4a6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:13.918521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:54:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:13.924554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d994-d6d1-42b2-bbd8-1d0c2b6b7b93 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:13.925036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d994-d6d1-42b2-bbd8-1d0c2b6b7b93 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:13.925153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d994-d6d1-42b2-bbd8-1d0c2b6b7b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:13.944239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4071d994-d6d1-42b2-bbd8-1d0c2b6b7b93 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:13.945436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4071d994-d6d1-42b2-bbd8-1d0c2b6b7b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:13.954872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:14.106778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dd057-e9c0-4df3-bc7e-7fcb9f90a4a6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:14.130462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dd057-e9c0-4df3-bc7e-7fcb9f90a4a6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:14.136946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c639727-eeb0-4afc-a92c-3f833e2d4e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:14.137587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c639727-eeb0-4afc-a92c-3f833e2d4e8e service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:14.137865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c639727-eeb0-4afc-a92c-3f833e2d4e8e service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:14.138136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c639727-eeb0-4afc-a92c-3f833e2d4e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:14.146918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c639727-eeb0-4afc-a92c-3f833e2d4e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:14.148563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c639727-eeb0-4afc-a92c-3f833e2d4e8e service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:14.158055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 721/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:16.084304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c00ce2-9846-4736-9b47-7fdd0ad6bf76 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:16.084906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c00ce2-9846-4736-9b47-7fdd0ad6bf76 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:16.084906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c00ce2-9846-4736-9b47-7fdd0ad6bf76 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:16.086254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:16.094826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e845-dd27-4428-843f-07515219439c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:16.095546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e845-dd27-4428-843f-07515219439c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:16.095721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e845-dd27-4428-843f-07515219439c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:16.117811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4944e845-dd27-4428-843f-07515219439c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:16.119296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4944e845-dd27-4428-843f-07515219439c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:16.128227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:16.172681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c00ce2-9846-4736-9b47-7fdd0ad6bf76 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:16.184942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c00ce2-9846-4736-9b47-7fdd0ad6bf76 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:16.188928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874c0f9-1a99-4bd2-9696-804fef4067a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:16.189604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874c0f9-1a99-4bd2-9696-804fef4067a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:16.189891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874c0f9-1a99-4bd2-9696-804fef4067a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:16.190342 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874c0f9-1a99-4bd2-9696-804fef4067a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:16.200086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874c0f9-1a99-4bd2-9696-804fef4067a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:16.201564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874c0f9-1a99-4bd2-9696-804fef4067a3 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:16.211888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 721/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 09:54:18.383587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47debd91-3fbd-4137-acdd-1854587fc986 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:18.384454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47debd91-3fbd-4137-acdd-1854587fc986 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:18.384454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47debd91-3fbd-4137-acdd-1854587fc986 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:18.386534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:18.393977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cb59a-48e8-45db-a6cb-a7c8e26b3b6c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:18.394485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cb59a-48e8-45db-a6cb-a7c8e26b3b6c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:18.394485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cb59a-48e8-45db-a6cb-a7c8e26b3b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:18.415130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-825cb59a-48e8-45db-a6cb-a7c8e26b3b6c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:18.416390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-825cb59a-48e8-45db-a6cb-a7c8e26b3b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:18.424752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:18.467625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47debd91-3fbd-4137-acdd-1854587fc986 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:18.481364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47debd91-3fbd-4137-acdd-1854587fc986 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:18.485101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93c4bb-29ae-4084-a6cb-db4eb14e6b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:18.485741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93c4bb-29ae-4084-a6cb-db4eb14e6b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:18.486071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93c4bb-29ae-4084-a6cb-db4eb14e6b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:18.486343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93c4bb-29ae-4084-a6cb-db4eb14e6b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:18.496071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93c4bb-29ae-4084-a6cb-db4eb14e6b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:18.497512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93c4bb-29ae-4084-a6cb-db4eb14e6b32 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:18.507582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 724/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:20.877398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09bfdc-440a-4ba0-8bf5-bb321343e99a None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:20.878002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09bfdc-440a-4ba0-8bf5-bb321343e99a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:20.878002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09bfdc-440a-4ba0-8bf5-bb321343e99a None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:20.879323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:20.885886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329861f5-024d-4082-ad21-21456aac056b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:20.886564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329861f5-024d-4082-ad21-21456aac056b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:20.886894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329861f5-024d-4082-ad21-21456aac056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:20.914384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-329861f5-024d-4082-ad21-21456aac056b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:20.915607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-329861f5-024d-4082-ad21-21456aac056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:20.925503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:20.968985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09bfdc-440a-4ba0-8bf5-bb321343e99a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:20.982563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09bfdc-440a-4ba0-8bf5-bb321343e99a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:20.987599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae63e8c-0b33-45cc-9ecf-0b193b7680e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:20.988310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae63e8c-0b33-45cc-9ecf-0b193b7680e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:20.988594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae63e8c-0b33-45cc-9ecf-0b193b7680e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:20.988862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae63e8c-0b33-45cc-9ecf-0b193b7680e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:20.998915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae63e8c-0b33-45cc-9ecf-0b193b7680e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:21.000415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae63e8c-0b33-45cc-9ecf-0b193b7680e6 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:21.010576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 724/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:23.149263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f772e7-9ce2-45da-9ea2-1fada437f87a None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:23.150147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f772e7-9ce2-45da-9ea2-1fada437f87a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:23.150147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f772e7-9ce2-45da-9ea2-1fada437f87a None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:23.152512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:23.157908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20eabb-fe4f-446f-bdab-72e3794a2c30 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:23.158442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20eabb-fe4f-446f-bdab-72e3794a2c30 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:23.158538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20eabb-fe4f-446f-bdab-72e3794a2c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:23.176575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8d20eabb-fe4f-446f-bdab-72e3794a2c30 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:23.177769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8d20eabb-fe4f-446f-bdab-72e3794a2c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:23.185984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:23.226075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f772e7-9ce2-45da-9ea2-1fada437f87a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:23.244445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f772e7-9ce2-45da-9ea2-1fada437f87a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:23.250846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc24745-5242-403d-86f2-c2f3a3a7edb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:23.251768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc24745-5242-403d-86f2-c2f3a3a7edb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:23.252347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc24745-5242-403d-86f2-c2f3a3a7edb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:23.252766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc24745-5242-403d-86f2-c2f3a3a7edb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:23.262420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc24745-5242-403d-86f2-c2f3a3a7edb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:23.263887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc24745-5242-403d-86f2-c2f3a3a7edb9 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:23.273736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 727/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:25.406349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa13b40-05db-4e66-ad1f-c8d3f8abf4cc None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:25.406910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa13b40-05db-4e66-ad1f-c8d3f8abf4cc None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:25.406910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa13b40-05db-4e66-ad1f-c8d3f8abf4cc None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:25.408472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:25.415518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c933b-6183-4389-bcc8-9ebd851fc552 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:25.416304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c933b-6183-4389-bcc8-9ebd851fc552 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:25.416462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c933b-6183-4389-bcc8-9ebd851fc552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:25.443540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a33c933b-6183-4389-bcc8-9ebd851fc552 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:25.444762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a33c933b-6183-4389-bcc8-9ebd851fc552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:25.453983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:25.495819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa13b40-05db-4e66-ad1f-c8d3f8abf4cc None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:25.510687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa13b40-05db-4e66-ad1f-c8d3f8abf4cc None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:25.514978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06c2eb-f675-4079-8ea7-62835ab7b712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:25.515614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06c2eb-f675-4079-8ea7-62835ab7b712 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:25.515925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06c2eb-f675-4079-8ea7-62835ab7b712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:25.516190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06c2eb-f675-4079-8ea7-62835ab7b712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:25.525656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06c2eb-f675-4079-8ea7-62835ab7b712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:25.527099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06c2eb-f675-4079-8ea7-62835ab7b712 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:25.536736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 727/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:27.651752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea300b0-58aa-4011-a5be-cb3780058e61 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:27.652673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea300b0-58aa-4011-a5be-cb3780058e61 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:27.652673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea300b0-58aa-4011-a5be-cb3780058e61 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:27.654898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:27.660719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3b13f-18cd-4ca5-ac5e-d77c53d609cf None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:27.661155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3b13f-18cd-4ca5-ac5e-d77c53d609cf None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:27.661252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3b13f-18cd-4ca5-ac5e-d77c53d609cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:27.680426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6df3b13f-18cd-4ca5-ac5e-d77c53d609cf None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:27.681624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6df3b13f-18cd-4ca5-ac5e-d77c53d609cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:27.690370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:27.732660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea300b0-58aa-4011-a5be-cb3780058e61 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:27.751953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea300b0-58aa-4011-a5be-cb3780058e61 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:27.755906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c52a6-8e1c-4e2e-9321-ce63dd83dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:27.756615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c52a6-8e1c-4e2e-9321-ce63dd83dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:27.756905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c52a6-8e1c-4e2e-9321-ce63dd83dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:27.757184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c52a6-8e1c-4e2e-9321-ce63dd83dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:27.767399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c52a6-8e1c-4e2e-9321-ce63dd83dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:27.768899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c52a6-8e1c-4e2e-9321-ce63dd83dbd5 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:27.779160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 730/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:29.868582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e431a2-4ff4-491e-82ef-48fe7140f208 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:29.869484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e431a2-4ff4-491e-82ef-48fe7140f208 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:29.869484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e431a2-4ff4-491e-82ef-48fe7140f208 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:29.871782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:29.878079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd45fc-50b1-46ed-9a9f-c6d3dc7c2ce0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:29.878529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd45fc-50b1-46ed-9a9f-c6d3dc7c2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:29.878637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd45fc-50b1-46ed-9a9f-c6d3dc7c2ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:29.904080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-48bd45fc-50b1-46ed-9a9f-c6d3dc7c2ce0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:29.905279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-48bd45fc-50b1-46ed-9a9f-c6d3dc7c2ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:29.913847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:29.954839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e431a2-4ff4-491e-82ef-48fe7140f208 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:29.972469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e431a2-4ff4-491e-82ef-48fe7140f208 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:29.976783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6175233-1c26-4ea5-b8a0-727e2aec2d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:29.977430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6175233-1c26-4ea5-b8a0-727e2aec2d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:29.977715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6175233-1c26-4ea5-b8a0-727e2aec2d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:29.977993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6175233-1c26-4ea5-b8a0-727e2aec2d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:29.988988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6175233-1c26-4ea5-b8a0-727e2aec2d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:29.990438 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6175233-1c26-4ea5-b8a0-727e2aec2d29 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:30.000840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 730/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:32.042868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a03d1-8a99-4c9d-9de3-2828d4aa7aa3 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:32.043564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a03d1-8a99-4c9d-9de3-2828d4aa7aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:32.043564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a03d1-8a99-4c9d-9de3-2828d4aa7aa3 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:32.044826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:54:32.051455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8829e37-c195-4865-aced-d42bb1f44067 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:32.052272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8829e37-c195-4865-aced-d42bb1f44067 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:32.052435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8829e37-c195-4865-aced-d42bb1f44067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:32.063148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=79 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:54:32.077414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e8829e37-c195-4865-aced-d42bb1f44067 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:32.078986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e8829e37-c195-4865-aced-d42bb1f44067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:32.088181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:32.129434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a03d1-8a99-4c9d-9de3-2828d4aa7aa3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:32.138482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=73 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:54:32.149578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a03d1-8a99-4c9d-9de3-2828d4aa7aa3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:32.156335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5db5f-760d-4c4f-98e9-f8b154c555b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:32.157360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5db5f-760d-4c4f-98e9-f8b154c555b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:32.157827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5db5f-760d-4c4f-98e9-f8b154c555b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:32.158256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5db5f-760d-4c4f-98e9-f8b154c555b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:32.168276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5db5f-760d-4c4f-98e9-f8b154c555b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:32.169738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5db5f-760d-4c4f-98e9-f8b154c555b2 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:32.179598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 733/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:34.338465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e512-76b8-4659-b618-ef0af90fc7e8 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:34.339292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e512-76b8-4659-b618-ef0af90fc7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:34.339408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e512-76b8-4659-b618-ef0af90fc7e8 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:34.341720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:34.347778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5607cd9-a62d-41d2-ae20-fca9d4249ffd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:34.348265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5607cd9-a62d-41d2-ae20-fca9d4249ffd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:34.348363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5607cd9-a62d-41d2-ae20-fca9d4249ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:34.367645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e5607cd9-a62d-41d2-ae20-fca9d4249ffd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:34.368857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e5607cd9-a62d-41d2-ae20-fca9d4249ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:34.378289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:34.419284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e512-76b8-4659-b618-ef0af90fc7e8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:34.438885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e512-76b8-4659-b618-ef0af90fc7e8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:34.443587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f759a27-e052-4129-9101-66033f39ca7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:34.444270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f759a27-e052-4129-9101-66033f39ca7f service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:34.444546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f759a27-e052-4129-9101-66033f39ca7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:34.444820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f759a27-e052-4129-9101-66033f39ca7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:34.454232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f759a27-e052-4129-9101-66033f39ca7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:34.455908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f759a27-e052-4129-9101-66033f39ca7f service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:34.465269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 733/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:54:36.528990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682639c-1d90-468e-ac25-ea7c4d4d92fc None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:36.529595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682639c-1d90-468e-ac25-ea7c4d4d92fc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:36.529595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682639c-1d90-468e-ac25-ea7c4d4d92fc None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:36.530946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:54:36.537423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc910c3-99fb-4525-8ea8-416524892928 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:36.538124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc910c3-99fb-4525-8ea8-416524892928 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:36.538307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc910c3-99fb-4525-8ea8-416524892928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:36.560270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4fc910c3-99fb-4525-8ea8-416524892928 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:36.561388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4fc910c3-99fb-4525-8ea8-416524892928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:36.570060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:36.609799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682639c-1d90-468e-ac25-ea7c4d4d92fc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:36.622007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682639c-1d90-468e-ac25-ea7c4d4d92fc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:36.625933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb6e6f-a237-4f05-8625-64cafaa7119c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:36.626596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb6e6f-a237-4f05-8625-64cafaa7119c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:36.626962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb6e6f-a237-4f05-8625-64cafaa7119c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:36.627240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb6e6f-a237-4f05-8625-64cafaa7119c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:36.636646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb6e6f-a237-4f05-8625-64cafaa7119c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:36.638098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb6e6f-a237-4f05-8625-64cafaa7119c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:36.648345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 736/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 09:54:46.553885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=11 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:54:46.643680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=14 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:54:56.374652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd280983-46d3-4651-b758-db91ee6a545c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:56.375256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd280983-46d3-4651-b758-db91ee6a545c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:56.375256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd280983-46d3-4651-b758-db91ee6a545c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:56.376944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 735/1471] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:56.383481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2a1ea-1053-4a90-924d-dd25b99e1a35 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:56.383987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2a1ea-1053-4a90-924d-dd25b99e1a35 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:56.384109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2a1ea-1053-4a90-924d-dd25b99e1a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:56.405621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-efe2a1ea-1053-4a90-924d-dd25b99e1a35 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:56.406819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-efe2a1ea-1053-4a90-924d-dd25b99e1a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:56.416269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 737/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:56.465370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f313b6a-766d-45e7-bfe3-fc8333086ddb None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:56.465650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f313b6a-766d-45e7-bfe3-fc8333086ddb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:56.465936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f313b6a-766d-45e7-bfe3-fc8333086ddb None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:56.467317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 736/1473] 10.222.0.16 () {54 vars in 962 bytes} [Thu Jun 30 09:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:56.473121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2c0e-13eb-4de5-8fcf-e96f26619dc8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:56.473563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2c0e-13eb-4de5-8fcf-e96f26619dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:56.474009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2c0e-13eb-4de5-8fcf-e96f26619dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:56.497693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4beb2c0e-13eb-4de5-8fcf-e96f26619dc8 None None] MFA Rules not processed for user `0a98b63a4ad14d7399938ffb9c6b018a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:56.498892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4beb2c0e-13eb-4de5-8fcf-e96f26619dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:56.506413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 738/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Jun 30 09:54:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:56.513777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f313b6a-766d-45e7-bfe3-fc8333086ddb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:56.525462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f313b6a-766d-45e7-bfe3-fc8333086ddb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:56.529325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c787a76-e722-4e93-8ec7-351bc5f28968 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:56.529955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c787a76-e722-4e93-8ec7-351bc5f28968 service glance] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:56.530230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c787a76-e722-4e93-8ec7-351bc5f28968 service glance] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:56.530495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c787a76-e722-4e93-8ec7-351bc5f28968 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:56.539852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c787a76-e722-4e93-8ec7-351bc5f28968 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:56.541381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c787a76-e722-4e93-8ec7-351bc5f28968 service glance] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:56.551145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 737/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:54:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 09:54:58.645389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20ac3-bb3c-490e-ad6f-9531b5b1ed94 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:58.645983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20ac3-bb3c-490e-ad6f-9531b5b1ed94 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:58.645983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20ac3-bb3c-490e-ad6f-9531b5b1ed94 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:58.647309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 739/1476] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:54:58.653861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d09b18-0a58-4290-abc6-51f1eefbbc00 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:58.654328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d09b18-0a58-4290-abc6-51f1eefbbc00 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:58.654431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d09b18-0a58-4290-abc6-51f1eefbbc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:58.680734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f8d09b18-0a58-4290-abc6-51f1eefbbc00 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:54:58.681917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f8d09b18-0a58-4290-abc6-51f1eefbbc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:54:58.691417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 738/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:54:58.715693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20ac3-bb3c-490e-ad6f-9531b5b1ed94 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:54:58.729854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20ac3-bb3c-490e-ad6f-9531b5b1ed94 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:54:58.734388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:54:58.735040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:54:58.735427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:54:58.735707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:54:58.736499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:54:58.738304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:54:58.738304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 09:54:58.738304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 09:54:58.739471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c824523-cb32-4d57-8aac-d9908b7ffb0d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:54:58.759128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 740/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 09:54:58 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:55:00.801908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245ad5b-353f-49be-8316-2062d2aa76f3 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:00.802507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245ad5b-353f-49be-8316-2062d2aa76f3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:00.802507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245ad5b-353f-49be-8316-2062d2aa76f3 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:00.804625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 739/1479] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:55:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:55:00.812578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab604ab-2aca-4466-88ab-2d72b92165d5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:00.813063 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab604ab-2aca-4466-88ab-2d72b92165d5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:00.813164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab604ab-2aca-4466-88ab-2d72b92165d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:00.832336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6ab604ab-2aca-4466-88ab-2d72b92165d5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:55:00.833538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6ab604ab-2aca-4466-88ab-2d72b92165d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:55:00.842299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 741/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:55:00.889946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245ad5b-353f-49be-8316-2062d2aa76f3 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:55:00.905742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245ad5b-353f-49be-8316-2062d2aa76f3 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:55:00.911667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a5b98-985d-477b-bc6f-40bd67428199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:55:00.912338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a5b98-985d-477b-bc6f-40bd67428199 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:00.912620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a5b98-985d-477b-bc6f-40bd67428199 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:00.912891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a5b98-985d-477b-bc6f-40bd67428199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:00.921148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a5b98-985d-477b-bc6f-40bd67428199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:55:00.922810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a5b98-985d-477b-bc6f-40bd67428199 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:55:00.932225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 740/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:55:03.139000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f716-7aac-4790-a598-c424a55053ee None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:03.139594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f716-7aac-4790-a598-c424a55053ee None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:03.139594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f716-7aac-4790-a598-c424a55053ee None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:03.141116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 742/1482] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:55:03.147282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f140ed-73fa-4376-a780-2316e3787ff9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:03.147835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f140ed-73fa-4376-a780-2316e3787ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:03.147835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f140ed-73fa-4376-a780-2316e3787ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:03.167774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-30f140ed-73fa-4376-a780-2316e3787ff9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:55:03.169059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-30f140ed-73fa-4376-a780-2316e3787ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:55:03.178052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 741/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:55:03.217865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59f716-7aac-4790-a598-c424a55053ee None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:55:03.229615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59f716-7aac-4790-a598-c424a55053ee None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:55:03.235143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf29ee9-8878-469e-844d-e604a466c392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:55:03.235865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf29ee9-8878-469e-844d-e604a466c392 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:03.236247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf29ee9-8878-469e-844d-e604a466c392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:03.236548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf29ee9-8878-469e-844d-e604a466c392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:03.246400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf29ee9-8878-469e-844d-e604a466c392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:55:03.247851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf29ee9-8878-469e-844d-e604a466c392 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:55:03.257818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 743/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:55:05.326401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812272b0-a4eb-41ec-b4b4-bbcafb9b3fc4 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:05.326914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812272b0-a4eb-41ec-b4b4-bbcafb9b3fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:05.326914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812272b0-a4eb-41ec-b4b4-bbcafb9b3fc4 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:05.328690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 742/1485] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:55:05.334245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1967-ccb1-4593-b93d-00fda69a1ba0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:05.334670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1967-ccb1-4593-b93d-00fda69a1ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:05.334777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1967-ccb1-4593-b93d-00fda69a1ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:05.354574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1f5e1967-ccb1-4593-b93d-00fda69a1ba0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:55:05.355784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1f5e1967-ccb1-4593-b93d-00fda69a1ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:55:05.365208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 744/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:55:05.407085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812272b0-a4eb-41ec-b4b4-bbcafb9b3fc4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:55:05.425241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812272b0-a4eb-41ec-b4b4-bbcafb9b3fc4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:55:05.429657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595720-1723-438a-b789-71145bf61d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:55:05.430309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595720-1723-438a-b789-71145bf61d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:05.430590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595720-1723-438a-b789-71145bf61d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:05.430865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595720-1723-438a-b789-71145bf61d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:05.440322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595720-1723-438a-b789-71145bf61d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:55:05.441770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595720-1723-438a-b789-71145bf61d63 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:55:05.451257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 743/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:55:07.588026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb029a5-a20d-4166-ab98-4c60ff81d407 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:07.588587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb029a5-a20d-4166-ab98-4c60ff81d407 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:07.588587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb029a5-a20d-4166-ab98-4c60ff81d407 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:07.590639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 745/1488] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:55:07.598800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b872e74-1648-4d40-9aa5-5c70f43f2fd0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:07.599243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b872e74-1648-4d40-9aa5-5c70f43f2fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:07.599371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b872e74-1648-4d40-9aa5-5c70f43f2fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:07.621780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0b872e74-1648-4d40-9aa5-5c70f43f2fd0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:55:07.622995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0b872e74-1648-4d40-9aa5-5c70f43f2fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:55:07.639423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 744/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:55:07.690693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb029a5-a20d-4166-ab98-4c60ff81d407 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:55:07.702954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb029a5-a20d-4166-ab98-4c60ff81d407 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:55:07.708647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980bd31-0266-4a2f-b21c-b630d7c7df5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:55:07.709283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bd31-0266-4a2f-b21c-b630d7c7df5c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:07.709559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bd31-0266-4a2f-b21c-b630d7c7df5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:07.709829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bd31-0266-4a2f-b21c-b630d7c7df5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:07.718759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980bd31-0266-4a2f-b21c-b630d7c7df5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:55:07.720317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980bd31-0266-4a2f-b21c-b630d7c7df5c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:55:07.729618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 746/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 09:55:09.927284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5adb6-2bcb-4dee-bd52-d58bd65c6145 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:09.928216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5adb6-2bcb-4dee-bd52-d58bd65c6145 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:09.928216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5adb6-2bcb-4dee-bd52-d58bd65c6145 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:09.930411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 745/1491] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 09:55:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:55:09.935786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73215cd-4086-46eb-a924-bef0ff2b080c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:09.936255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73215cd-4086-46eb-a924-bef0ff2b080c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:09.936361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73215cd-4086-46eb-a924-bef0ff2b080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:09.957845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f73215cd-4086-46eb-a924-bef0ff2b080c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:55:09.959047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f73215cd-4086-46eb-a924-bef0ff2b080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:55:09.968347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 747/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:55:10.007216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5adb6-2bcb-4dee-bd52-d58bd65c6145 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:55:10.018832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5adb6-2bcb-4dee-bd52-d58bd65c6145 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:55:10.022635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920418-4f95-4eb5-baa2-4378c3646249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:55:10.023265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920418-4f95-4eb5-baa2-4378c3646249 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:55:10.023539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920418-4f95-4eb5-baa2-4378c3646249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:55:10.023807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920418-4f95-4eb5-baa2-4378c3646249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:55:10.033738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920418-4f95-4eb5-baa2-4378c3646249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:55:10.035171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920418-4f95-4eb5-baa2-4378c3646249 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:55:10.044579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 746/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 09:55:19.950414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=33 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:55:20.040361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=33 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:56:43.074954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.075592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.075592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.096823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:56:43.097999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:43.107516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 748/1494] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:56:43.116500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.117013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.117013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.138396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:56:43.139579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:43.149026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 747/1495] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 09:56:43.153674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.164069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a918657-cab9-45ef-aefe-c25e99ae5486 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.169398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.170059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.170345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.170638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.171736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 09:56:43.171924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.172907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.172907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 09:56:43.172907 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 09:56:43.173793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.185829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 749/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 09:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:56:43.197981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.209605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa05e6-e38d-4128-98cf-fb886eb1f334 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.215352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.216057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.216336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.216600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.217279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.219044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.219044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 09:56:43.219044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 09:56:43.220273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.268050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 748/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 09:56:43 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:56:43.272532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.283169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf0626-9ad9-4eab-97f7-47c02d245d3d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.343831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.344535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.344815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.345089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.345785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.346815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.346815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 09:56:43.346815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 09:56:43.348039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.369916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 750/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 09:56:43 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 09:56:43.374299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.384613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cacb9f-e403-47cc-9943-74ca14c68874 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.389206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.390056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.390291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.390666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.391400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.392449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.392449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 09:56:43.392449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 09:56:43.393336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.406228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 749/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 09:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 09:56:43.410094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.419863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e761a2b-4faa-4ce0-95a4-68a65e3e96e7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.424719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.425370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.425647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.425926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.426648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.427658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.427658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 09:56:43.427658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 09:56:43.428557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29d0fb-da10-481a-8e94-58a52f120f21 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.441191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 751/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 09:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 09:56:43.445261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.455519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2bc3-46d4-4da4-ae0e-542197f2eec2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.462007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.463288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.463680 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.464087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] PATH_INFO: `/v3/projects/3c88b54771c54a328c68a3941149b85a/users/28adcaddb78a457ca7a65fc63cd2cdf6/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.486459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c88b54771c54a328c68a3941149b85a, user_id=28adcaddb78a457ca7a65fc63cd2cdf6, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.487539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.487539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 09:56:43.487539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 09:56:43.489501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43512e0-404e-4f0c-bdb0-a600bf46cffa admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.506419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 750/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 09:56:43 2022] PUT /identity/v3/projects/3c88b54771c54a328c68a3941149b85a/users/28adcaddb78a457ca7a65fc63cd2cdf6/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:56:43.518844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2044-51f0-4e5d-bec8-16e10a03a28b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.519131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2044-51f0-4e5d-bec8-16e10a03a28b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.519407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2044-51f0-4e5d-bec8-16e10a03a28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.545967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-43ec2044-51f0-4e5d-bec8-16e10a03a28b None None] MFA Rules not processed for user `28adcaddb78a457ca7a65fc63cd2cdf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:56:43.547472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-43ec2044-51f0-4e5d-bec8-16e10a03a28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:43.616412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 752/1502] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:56:43.630602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a140-2927-46d5-92a3-b140de7f955b None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.630855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a140-2927-46d5-92a3-b140de7f955b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.631190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a140-2927-46d5-92a3-b140de7f955b None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.632869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 751/1503] 10.222.0.16 () {54 vars in 963 bytes} [Thu Jun 30 09:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:56:43.637963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a87c0-3849-4353-ab0e-d9e96ca38e6a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.638250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a87c0-3849-4353-ab0e-d9e96ca38e6a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.638528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a87c0-3849-4353-ab0e-d9e96ca38e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.658650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9d2a87c0-3849-4353-ab0e-d9e96ca38e6a None None] MFA Rules not processed for user `15924f58d53145ae83b3b4cfbbb02ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:56:43.659904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9d2a87c0-3849-4353-ab0e-d9e96ca38e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:43.706420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:56:43.718306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 753/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Jun 30 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:56:43.726179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a140-2927-46d5-92a3-b140de7f955b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.737385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a140-2927-46d5-92a3-b140de7f955b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.741239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.741872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.742148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.742420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.752254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.753722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.762801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 752/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:56:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 09:56:43.809550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f3447-41b7-4eba-b8c7-ef9bf5890c17 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.809862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f3447-41b7-4eba-b8c7-ef9bf5890c17 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.810135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f3447-41b7-4eba-b8c7-ef9bf5890c17 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.811504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 754/1506] 10.222.0.16 () {52 vars in 829 bytes} [Thu Jun 30 09:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:56:43.815716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:43.882965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937043a-4f19-4113-9d54-68dd9b126052 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:43.887683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:43.888591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:43.888873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:43.889150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] PATH_INFO: `/v3/users/28adcaddb78a457ca7a65fc63cd2cdf6` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:43.890414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=28adcaddb78a457ca7a65fc63cd2cdf6)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:43.891466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:43.891466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 09:56:43.891466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 09:56:43.892688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:43.967159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28adcaddb78a457ca7a65fc63cd2cdf6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:43.967835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28adcaddb78a457ca7a65fc63cd2cdf6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:43.968603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28adcaddb78a457ca7a65fc63cd2cdf6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:43.981315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28adcaddb78a457ca7a65fc63cd2cdf6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.000420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28adcaddb78a457ca7a65fc63cd2cdf6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.009314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:56:44.023883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 753/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 09:56:43 2022] DELETE /identity/v3/users/28adcaddb78a457ca7a65fc63cd2cdf6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:56:44.028740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:44.029031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:44.029310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:44.051760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:56:44.052990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:44.114505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 755/1508] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 09:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:56:44.124277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:44.125940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:44.189021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09711b7-368e-49ae-b572-053b7cd24e3e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:44.192998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc1bf8-2afb-477b-9278-69c953aff929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:44.193640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc1bf8-2afb-477b-9278-69c953aff929 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:44.193917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc1bf8-2afb-477b-9278-69c953aff929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:44.194166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc1bf8-2afb-477b-9278-69c953aff929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:44.204097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc1bf8-2afb-477b-9278-69c953aff929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:44.205543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc1bf8-2afb-477b-9278-69c953aff929 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:44.214878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 754/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:56:44.560976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:56:44.562621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:56:44.574494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07085005-8b00-4996-8eb8-189fb1d825ff None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:56:44.580691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:56:44.581445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:56:44.581718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:56:44.581987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] PATH_INFO: `/v3/projects/3c88b54771c54a328c68a3941149b85a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:56:44.583249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c88b54771c54a328c68a3941149b85a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:56:44.589677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 09:56:44.589677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 09:56:44.589677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 09:56:44.590899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:56:44.654684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c88b54771c54a328c68a3941149b85a', 'request_id': 'req-072d97e1-2022-445d-9c3b-c840fa795908'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.660779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c88b54771c54a328c68a3941149b85a', 'request_id': 'req-072d97e1-2022-445d-9c3b-c840fa795908'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.673910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c88b54771c54a328c68a3941149b85a', 'request_id': 'req-072d97e1-2022-445d-9c3b-c840fa795908'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.694987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] The token cache is being invalidate because project 3c88b54771c54a328c68a3941149b85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 09:56:44.695246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.696271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-072d97e1-2022-445d-9c3b-c840fa795908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 09:56:44.698726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 756/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 09:56:44 2022] DELETE /identity/v3/projects/3c88b54771c54a328c68a3941149b85a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 09:56:54.210719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=29 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:56:54.689610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:58:23.020671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b03f1-66b4-4f7b-af8b-53f390def14f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:23.021313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b03f1-66b4-4f7b-af8b-53f390def14f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:23.021313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b03f1-66b4-4f7b-af8b-53f390def14f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:23.022761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 09:58:23.028722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef52a-bdc2-4a79-a397-cc7e18d93273 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:23.029195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef52a-bdc2-4a79-a397-cc7e18d93273 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:23.029303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef52a-bdc2-4a79-a397-cc7e18d93273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:23.052434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-eb9ef52a-bdc2-4a79-a397-cc7e18d93273 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:23.053724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eb9ef52a-bdc2-4a79-a397-cc7e18d93273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:23.124317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:23.277385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b03f1-66b4-4f7b-af8b-53f390def14f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:23.280268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ef1b03f1-66b4-4f7b-af8b-53f390def14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:23.370376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b03f1-66b4-4f7b-af8b-53f390def14f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:23.376552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11259-322e-45f6-adcb-d0b61b69af0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:23.377231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11259-322e-45f6-adcb-d0b61b69af0a service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:23.377523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11259-322e-45f6-adcb-d0b61b69af0a service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:23.377796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11259-322e-45f6-adcb-d0b61b69af0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:23.387055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c11259-322e-45f6-adcb-d0b61b69af0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:23.388769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c11259-322e-45f6-adcb-d0b61b69af0a service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:23.398808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 756/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:24.238952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420820b-140e-4e36-b0f6-f8aed9fff47e None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:24.239892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420820b-140e-4e36-b0f6-f8aed9fff47e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:24.239892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420820b-140e-4e36-b0f6-f8aed9fff47e None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:24.242331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:24.248332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a937f2-d991-436c-9f8d-b73f87c8f282 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:24.249005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a937f2-d991-436c-9f8d-b73f87c8f282 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:24.249161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a937f2-d991-436c-9f8d-b73f87c8f282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:24.270492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-30a937f2-d991-436c-9f8d-b73f87c8f282 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:24.271991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-30a937f2-d991-436c-9f8d-b73f87c8f282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:24.280476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:24.437317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420820b-140e-4e36-b0f6-f8aed9fff47e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:24.454308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420820b-140e-4e36-b0f6-f8aed9fff47e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:24.460115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54b22-5aa9-4ccc-ac0a-31e17602110e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:24.460751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54b22-5aa9-4ccc-ac0a-31e17602110e service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:24.461030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54b22-5aa9-4ccc-ac0a-31e17602110e service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:24.461299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54b22-5aa9-4ccc-ac0a-31e17602110e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:24.471149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54b22-5aa9-4ccc-ac0a-31e17602110e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:24.471961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54b22-5aa9-4ccc-ac0a-31e17602110e service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:24.482292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 759/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:24.854740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2d459-b305-4a02-94b9-7de8373c2f3d None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:24.855192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2d459-b305-4a02-94b9-7de8373c2f3d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:24.855662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2d459-b305-4a02-94b9-7de8373c2f3d None None] PATH_INFO: `/v3` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:24.857657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 758/1517] 10.222.0.13 () {54 vars in 925 bytes} [Thu Jun 30 09:58:24 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 09:58:24.893868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077ba47-1ae5-4d61-bd07-c05ee345ace7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:24.894159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077ba47-1ae5-4d61-bd07-c05ee345ace7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:24.894451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077ba47-1ae5-4d61-bd07-c05ee345ace7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:24.916140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3077ba47-1ae5-4d61-bd07-c05ee345ace7 None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:24.917354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3077ba47-1ae5-4d61-bd07-c05ee345ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:24.985548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 760/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 30 09:58:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:28.809898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f8159-3d75-4051-aeee-31a73682955c None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:28.810754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f8159-3d75-4051-aeee-31a73682955c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:28.810754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f8159-3d75-4051-aeee-31a73682955c None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:28.812874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 759/1519] 10.222.0.16 () {54 vars in 960 bytes} [Thu Jun 30 09:58:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:28.821161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede440-01ef-4773-885d-4359afca05cc None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:28.821586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede440-01ef-4773-885d-4359afca05cc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:28.821732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede440-01ef-4773-885d-4359afca05cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:28.841749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8dede440-01ef-4773-885d-4359afca05cc None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:28.842940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8dede440-01ef-4773-885d-4359afca05cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:28.852207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 761/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Thu Jun 30 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:28.859902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f8159-3d75-4051-aeee-31a73682955c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:28.876137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f8159-3d75-4051-aeee-31a73682955c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:28.881441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cb73-d29c-4271-b344-34afa544c37b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:28.882072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cb73-d29c-4271-b344-34afa544c37b service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:28.882352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cb73-d29c-4271-b344-34afa544c37b service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:28.882624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cb73-d29c-4271-b344-34afa544c37b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:28.891022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549cb73-d29c-4271-b344-34afa544c37b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:28.892650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549cb73-d29c-4271-b344-34afa544c37b service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:28.902363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 760/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:35.739919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b9e8f-da7d-4410-8bfb-95b7b840382d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:35.740777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b9e8f-da7d-4410-8bfb-95b7b840382d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:35.740777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b9e8f-da7d-4410-8bfb-95b7b840382d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:35.743112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:35.750690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ce35c-335a-4636-af65-1ec3c8f03ef1 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:35.751187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ce35c-335a-4636-af65-1ec3c8f03ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:35.751324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ce35c-335a-4636-af65-1ec3c8f03ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:35.778965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-859ce35c-335a-4636-af65-1ec3c8f03ef1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:35.780209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-859ce35c-335a-4636-af65-1ec3c8f03ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:35.789667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:35.950269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b9e8f-da7d-4410-8bfb-95b7b840382d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:35.975185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b9e8f-da7d-4410-8bfb-95b7b840382d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:35.982182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80671bc-be75-40a0-917a-7259754e1798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:35.982903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80671bc-be75-40a0-917a-7259754e1798 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:35.983258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80671bc-be75-40a0-917a-7259754e1798 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:35.983554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80671bc-be75-40a0-917a-7259754e1798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:35.993983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80671bc-be75-40a0-917a-7259754e1798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:35.995731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80671bc-be75-40a0-917a-7259754e1798 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:36.006533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 763/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:36.823421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c821fb-ff13-4dba-8b7e-55691cb5eae6 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:36.824052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c821fb-ff13-4dba-8b7e-55691cb5eae6 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:36.824052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c821fb-ff13-4dba-8b7e-55691cb5eae6 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:36.825902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 09:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:36.831205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7bbf7-528e-4117-906e-5a661169d905 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:36.831708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7bbf7-528e-4117-906e-5a661169d905 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:36.831818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7bbf7-528e-4117-906e-5a661169d905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:36.851146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-74c7bbf7-528e-4117-906e-5a661169d905 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:36.852418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-74c7bbf7-528e-4117-906e-5a661169d905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:36.861711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 09:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:37.035112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c821fb-ff13-4dba-8b7e-55691cb5eae6 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:37.055093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c821fb-ff13-4dba-8b7e-55691cb5eae6 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:37.062074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f900b6-e0b9-41b0-aac2-1f308b89dd83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:37.062751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f900b6-e0b9-41b0-aac2-1f308b89dd83 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:37.063020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f900b6-e0b9-41b0-aac2-1f308b89dd83 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:37.063306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f900b6-e0b9-41b0-aac2-1f308b89dd83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:37.072264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f900b6-e0b9-41b0-aac2-1f308b89dd83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:37.073997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f900b6-e0b9-41b0-aac2-1f308b89dd83 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:37.084581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 763/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:40.879974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d061c13-0b66-4dba-a8be-e1828770558c None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:40.880582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d061c13-0b66-4dba-a8be-e1828770558c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:40.880582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d061c13-0b66-4dba-a8be-e1828770558c None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:40.882760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:58:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:40.892739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eb2cd-515c-475e-9d16-4363e4a718db None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:40.893157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eb2cd-515c-475e-9d16-4363e4a718db None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:40.893331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eb2cd-515c-475e-9d16-4363e4a718db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:40.917271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-632eb2cd-515c-475e-9d16-4363e4a718db None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:40.918508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-632eb2cd-515c-475e-9d16-4363e4a718db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:40.927776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:40.966462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d061c13-0b66-4dba-a8be-e1828770558c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:40.969134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2d061c13-0b66-4dba-a8be-e1828770558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:41.030462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d061c13-0b66-4dba-a8be-e1828770558c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:41.035718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047da34-b4c8-46d2-81c9-835f5230ec61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:41.036504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047da34-b4c8-46d2-81c9-835f5230ec61 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:41.036779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047da34-b4c8-46d2-81c9-835f5230ec61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:41.037075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047da34-b4c8-46d2-81c9-835f5230ec61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:41.053196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047da34-b4c8-46d2-81c9-835f5230ec61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:41.055522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047da34-b4c8-46d2-81c9-835f5230ec61 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:41.063700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:58:41.069714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 766/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:43.109953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6646619-60ec-4ce7-b8fe-b6811bf72fb9 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:43.110838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6646619-60ec-4ce7-b8fe-b6811bf72fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:43.110838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6646619-60ec-4ce7-b8fe-b6811bf72fb9 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:43.113368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:58:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:43.118900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1e054-3cd5-47f8-b4eb-54f7abe921e3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:43.119322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1e054-3cd5-47f8-b4eb-54f7abe921e3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:43.119415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1e054-3cd5-47f8-b4eb-54f7abe921e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:43.139919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-67e1e054-3cd5-47f8-b4eb-54f7abe921e3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:43.141104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-67e1e054-3cd5-47f8-b4eb-54f7abe921e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:43.150165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:43.186487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6646619-60ec-4ce7-b8fe-b6811bf72fb9 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:43.200494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6646619-60ec-4ce7-b8fe-b6811bf72fb9 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:43.204946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919df76-41b4-4e49-839e-d048785cf257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:43.205586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919df76-41b4-4e49-839e-d048785cf257 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:43.205869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919df76-41b4-4e49-839e-d048785cf257 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:43.206144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919df76-41b4-4e49-839e-d048785cf257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:43.216136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919df76-41b4-4e49-839e-d048785cf257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:43.217580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919df76-41b4-4e49-839e-d048785cf257 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:43.227488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 766/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:46.917693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79062dbe-2baa-48ad-9271-cff6ac1e89b8 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:46.918581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79062dbe-2baa-48ad-9271-cff6ac1e89b8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:46.918581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79062dbe-2baa-48ad-9271-cff6ac1e89b8 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:46.920807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:58:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:46.929616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a402-e7fa-4bb5-aa41-837f86e965ed None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:46.930034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a402-e7fa-4bb5-aa41-837f86e965ed None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:46.930195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a402-e7fa-4bb5-aa41-837f86e965ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:46.940473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=62 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:58:46.954457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e4b2a402-e7fa-4bb5-aa41-837f86e965ed None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:46.955638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e4b2a402-e7fa-4bb5-aa41-837f86e965ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:46.965000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:47.270961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79062dbe-2baa-48ad-9271-cff6ac1e89b8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:47.272973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-79062dbe-2baa-48ad-9271-cff6ac1e89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:47.334508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79062dbe-2baa-48ad-9271-cff6ac1e89b8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:47.338766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b93967-1c50-4e29-b488-ddfc2dd091c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:47.339411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b93967-1c50-4e29-b488-ddfc2dd091c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:47.339684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b93967-1c50-4e29-b488-ddfc2dd091c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:47.340000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b93967-1c50-4e29-b488-ddfc2dd091c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:47.349646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b93967-1c50-4e29-b488-ddfc2dd091c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:47.351153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b93967-1c50-4e29-b488-ddfc2dd091c3 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:47.360412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 769/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:49.398124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e02b83-2c1a-4dc6-946b-60a49b101286 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:49.398601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e02b83-2c1a-4dc6-946b-60a49b101286 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:49.398715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e02b83-2c1a-4dc6-946b-60a49b101286 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:49.400133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 09:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 09:58:49.406420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce41e-6d1f-431c-a065-b83ed6302af1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:49.406935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce41e-6d1f-431c-a065-b83ed6302af1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:49.407071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ce41e-6d1f-431c-a065-b83ed6302af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:49.425887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b28ce41e-6d1f-431c-a065-b83ed6302af1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 09:58:49.427099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b28ce41e-6d1f-431c-a065-b83ed6302af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:49.435995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 09:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 09:58:49.739694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e02b83-2c1a-4dc6-946b-60a49b101286 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 09:58:49.742002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a7e02b83-2c1a-4dc6-946b-60a49b101286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 09:58:49.757626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e02b83-2c1a-4dc6-946b-60a49b101286 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 09:58:49.761619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b77aa-8698-4c8f-bced-2dfbd8446e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 09:58:49.762280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b77aa-8698-4c8f-bced-2dfbd8446e38 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 09:58:49.762590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b77aa-8698-4c8f-bced-2dfbd8446e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 09:58:49.762867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b77aa-8698-4c8f-bced-2dfbd8446e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 09:58:49.772534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b77aa-8698-4c8f-bced-2dfbd8446e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 09:58:49.773965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b77aa-8698-4c8f-bced-2dfbd8446e38 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 09:58:49.783598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 769/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 09:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 09:58:59.420663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=32 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 09:58:59.779501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=8 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:19.079300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.080184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.080184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.103050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.104302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.118841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 771/1540] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:19.128641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.129508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.129508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.153829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.155048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.166749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 770/1541] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.171738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.185317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849fff-627c-4e94-b2bd-c65fdce690e4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.193660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.194360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.194636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.197202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.197202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.197202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.198334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.216066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 772/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.224695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.237917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96f12a-b2d8-4fcd-b892-315c1973b51d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.246188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f454b-d770-428e-af79-b04584e61368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.246910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f454b-d770-428e-af79-b04584e61368 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.247454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f454b-d770-428e-af79-b04584e61368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.247741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f454b-d770-428e-af79-b04584e61368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.248487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f454b-d770-428e-af79-b04584e61368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.250466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f454b-d770-428e-af79-b04584e61368 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.300174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 771/1543] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:19.304257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.312464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebf00d-43cf-4f9e-b7a6-f91cd990c8ef None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.312787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebf00d-43cf-4f9e-b7a6-f91cd990c8ef None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.313021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebf00d-43cf-4f9e-b7a6-f91cd990c8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.317909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33ab6-ce0e-4816-b901-44fb147d66f3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.356770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-61ebf00d-43cf-4f9e-b7a6-f91cd990c8ef None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.362972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-61ebf00d-43cf-4f9e-b7a6-f91cd990c8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.469581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f138ef7-f5ab-4f43-849e-8d978c508e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.470288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138ef7-f5ab-4f43-849e-8d978c508e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.470573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138ef7-f5ab-4f43-849e-8d978c508e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.470852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138ef7-f5ab-4f43-849e-8d978c508e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.475891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f138ef7-f5ab-4f43-849e-8d978c508e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.475891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f138ef7-f5ab-4f43-849e-8d978c508e41 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.501198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 772/1544] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.504527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 773/1545] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:19.504689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd21ab-353f-4f9f-bdbd-842326ca34ab None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.504689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd21ab-353f-4f9f-bdbd-842326ca34ab None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.504873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e789a-4467-421c-aa29-c3c61c463f9c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.504982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd21ab-353f-4f9f-bdbd-842326ca34ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.505078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e789a-4467-421c-aa29-c3c61c463f9c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.505078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e789a-4467-421c-aa29-c3c61c463f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.522529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0ccd21ab-353f-4f9f-bdbd-842326ca34ab None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.523748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0ccd21ab-353f-4f9f-bdbd-842326ca34ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.527536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1f5e789a-4467-421c-aa29-c3c61c463f9c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.528802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1f5e789a-4467-421c-aa29-c3c61c463f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.532620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 773/1546] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.533761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f78f0e-a0b7-43dd-98c5-31e3b98bdaad None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.534106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f78f0e-a0b7-43dd-98c5-31e3b98bdaad None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.534337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f78f0e-a0b7-43dd-98c5-31e3b98bdaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.536981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 774/1547] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.538338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.538631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.538916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.553559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e3f78f0e-a0b7-43dd-98c5-31e3b98bdaad None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.554757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e3f78f0e-a0b7-43dd-98c5-31e3b98bdaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.563222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 774/1548] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.563782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.564523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f388796-f29f-40a3-9b1f-02b267e9aa97 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.564811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f388796-f29f-40a3-9b1f-02b267e9aa97 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.565090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f388796-f29f-40a3-9b1f-02b267e9aa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.565485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.577787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 775/1549] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.578893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.592096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac8f1-7e43-41e0-8ca5-e0f84c9c95c5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.594486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0f388796-f29f-40a3-9b1f-02b267e9aa97 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.595715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0f388796-f29f-40a3-9b1f-02b267e9aa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.597305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a9bc7-f9bd-436d-a150-8d8033a394d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.598076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a9bc7-f9bd-436d-a150-8d8033a394d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.598447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a9bc7-f9bd-436d-a150-8d8033a394d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.598942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a9bc7-f9bd-436d-a150-8d8033a394d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.599697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a9bc7-f9bd-436d-a150-8d8033a394d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.601511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a9bc7-f9bd-436d-a150-8d8033a394d4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.603907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 775/1550] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.605099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817d424-2681-46b2-9b4a-d3b9b0ba994a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.605431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817d424-2681-46b2-9b4a-d3b9b0ba994a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.606019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817d424-2681-46b2-9b4a-d3b9b0ba994a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.616926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 776/1551] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:19.618487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4902ec-c780-4bd1-bbc4-21990a6b83d4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.618778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4902ec-c780-4bd1-bbc4-21990a6b83d4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.619055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4902ec-c780-4bd1-bbc4-21990a6b83d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.623463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a817d424-2681-46b2-9b4a-d3b9b0ba994a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.624710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a817d424-2681-46b2-9b4a-d3b9b0ba994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.632603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 776/1552] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.633840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df8888-2db2-4f97-889d-57e7d72de64e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.634121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df8888-2db2-4f97-889d-57e7d72de64e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.634395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df8888-2db2-4f97-889d-57e7d72de64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.640161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5c4902ec-c780-4bd1-bbc4-21990a6b83d4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.642547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5c4902ec-c780-4bd1-bbc4-21990a6b83d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.651330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 777/1553] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.652469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-01df8888-2db2-4f97-889d-57e7d72de64e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.653122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16caa3e-5e52-4556-a368-8c1c9d8df35d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.653407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16caa3e-5e52-4556-a368-8c1c9d8df35d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.653684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16caa3e-5e52-4556-a368-8c1c9d8df35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.653889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-01df8888-2db2-4f97-889d-57e7d72de64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.661928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 777/1554] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.663318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261534f-259f-4abd-b736-7cbb3a844cab None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.663657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261534f-259f-4abd-b736-7cbb3a844cab None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.664030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261534f-259f-4abd-b736-7cbb3a844cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.675743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a16caa3e-5e52-4556-a368-8c1c9d8df35d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.676968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a16caa3e-5e52-4556-a368-8c1c9d8df35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.682217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a261534f-259f-4abd-b736-7cbb3a844cab None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.684676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a261534f-259f-4abd-b736-7cbb3a844cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.685297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 778/1555] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.686608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.686873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.687134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.693572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 778/1556] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.694619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.694903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.695175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.708522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.709734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.713556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.714914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.718047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 779/1557] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.718816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.728348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4fe6e-3393-47bb-8e73-cecfe8844594 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.729301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 779/1558] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.730299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.733578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.734324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.734670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.735156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.735804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.737450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.739884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a88a86-97f4-4ea5-afee-457f0587b435 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.744333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.744985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.745263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.745490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.746305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.746612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.747640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:00:19.747640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 10:00:19.747640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 10:00:19.748572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.748719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 780/1559] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:19.749436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.758790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe8ea9-86b0-4adb-82c4-8d8da2082122 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.760713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 780/1560] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.767496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.770083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.770738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.771015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.771285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.772093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.772395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.774040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.777464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8890085-4e6e-4f68-bd7a-67846d2862fe admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.781896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.782904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.783201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.783469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.784230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.784532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.786124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 781/1561] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.786944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.788192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.796137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2283e9-e4e2-466e-b69c-774a80ed16ad admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.800663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.801305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.801585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.801691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 781/1562] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.801856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.802559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.802671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.802944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.804526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5d5fb-570d-4bee-a23b-335a25376617 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.813746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b5a2-eb48-4032-904a-bcbe3c4e93f7 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.823517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 782/1563] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.823517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.823517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.823517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.824132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.824132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.824132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.824132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.824652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.824652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.824652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.835617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 782/1564] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.836216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:19.837042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.837411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:19.845444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 783/1565] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:19.846198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.851485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ca0a2-84d4-46ba-ab1e-84c7e67725e5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.855338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f7fa4-5464-47ca-8bfe-c2b01a568fe2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.856526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.857193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.857476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.857755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.858491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:19.858801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.859970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.860450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.860840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.861116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.861388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] PATH_INFO: `/v3/projects/0e8bc4e999ce4f1e9f52b456bcdd88d3/users/a179e34636ed4970be4f0790115f9d8f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.872677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 783/1566] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:19.873601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.876791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8bc4e999ce4f1e9f52b456bcdd88d3, user_id=a179e34636ed4970be4f0790115f9d8f, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.877837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:00:19.877837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 10:00:19.877837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 10:00:19.879753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.884727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce762-504d-4b56-81a7-c0d05f7cf6ff admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.890004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.890654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.890933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.891205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.891921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.893834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:19.893989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 784/1567] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:19 2022] PUT /identity/v3/projects/0e8bc4e999ce4f1e9f52b456bcdd88d3/users/a179e34636ed4970be4f0790115f9d8f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:19.894805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.913829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:19.923315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 784/1568] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:19.924364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:19.938940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:19.964403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57931bf6-3c3a-472e-8517-6f5fac45f7ee admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.988494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde95322-6420-4013-84f6-afa10872f0f8 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:19.993260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:19.993902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:19.994346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:19.994538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:19.995149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:19.998307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.024369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.025171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.025549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 785/1569] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.025690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.025898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.026180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.026675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.027803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:00:20.027803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 10:00:20.027803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 10:00:20.029172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.057371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 785/1570] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:19 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.057371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.095480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868dc2c-2fe2-43d3-85e6-1f90f88588c1 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.125669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a2a7f-3313-41e5-a76e-7abc11192248 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.130461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.131171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.131451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.131718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.132501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.134442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.157753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.158454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.158734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.159013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.159711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.161243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 786/1571] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.161664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.162148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.187619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 786/1572] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.188596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.202856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:20.218575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:20.227102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0d28-0a94-4f39-9b57-9b290bcbea66 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.250565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4184a44-dc43-4f93-a4ac-2ddc0f7c8ff2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.255310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7110b-5ad5-4a4d-991f-64818d7f991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.256037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7110b-5ad5-4a4d-991f-64818d7f991c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.256317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7110b-5ad5-4a4d-991f-64818d7f991c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.256591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7110b-5ad5-4a4d-991f-64818d7f991c admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.257293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7110b-5ad5-4a4d-991f-64818d7f991c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.259181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7110b-5ad5-4a4d-991f-64818d7f991c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.287194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 787/1573] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.288709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.288997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.289272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.297837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.298751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.299241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.299711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.301102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:20.301635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.304636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.311413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:20.312669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:20.318329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 787/1574] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:20.318941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.330519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8f13d-bf0f-4c11-ae30-a7924d3bc9f6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.390189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 788/1575] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:20.390189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.397052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01f41c-fa4f-4415-aedb-0ab1cc53ca73 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.398443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.399127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.399409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.399687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.400453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.402669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.402833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.403502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.403772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.404092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.404773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.405798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:00:20.405798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 10:00:20.405798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 10:00:20.407013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.430949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 788/1576] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.430949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.431259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 789/1577] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 283 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.431259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.435893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa519e-7d7a-4c6e-9648-a7f0d7d6ac92 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.440812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c5de8-b0e6-4a07-8f91-2614a7da56a3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.441693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.442398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.442675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.442948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.443638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.445435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.445595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.446093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.446376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.446651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.447675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.449813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.467189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 789/1578] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.468105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.470161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 790/1579] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.472876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.478151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222df28-df02-4ff3-8790-2edc0236c77d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.482686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.483343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.483480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c2bc6-bc97-41ff-b75f-96ba4cbd635c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.483624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.483925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.484618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.486536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.488836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.489488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.489765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.490032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.490712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.492669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.509026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 790/1580] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.510079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.518195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 791/1581] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.519157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.520726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2524-253a-43fd-86b4-6071a456e885 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.521257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:20.525644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.526315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.526667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.527044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.527946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.529956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.556982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 791/1582] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.558351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.567758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:20.584976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec415c-1470-4e70-9272-95c82a0ea526 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.597900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:20.621228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2d382-277f-4190-957b-b6de3f9896fe admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.625838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.626509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.626788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.627061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.627797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.629446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.636078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.636793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.637074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.637352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.642439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 792/1583] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.643676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.649020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.650514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.656180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed9008-ede5-418a-91c9-b35334e949c7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.660824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 792/1584] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:20.661495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.662313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.663066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.663349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.663625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.664447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.666183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.673324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fdf89-6d17-46ef-bae4-c5847d0e679b service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.682136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 793/1585] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.682373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.682509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.682620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.682620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.682620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.682620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.683514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.695121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835e46c-9df3-4d1f-9393-5e2a705259c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.696175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 793/1586] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.696701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.704137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.704827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.705108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.705380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.706114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.707744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.709392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f30628-0416-45c3-90fd-aecfc0f48578 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.714376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.715655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.716301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.716587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.717583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.719255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.720468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 794/1587] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.721633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.733966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05825e60-3234-4f13-a79d-fbcaf6d8ed19 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.738112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 794/1588] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.739186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.740720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.742008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.742419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.742702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.743497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.745350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.751491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744721-32ee-4317-b701-2ba9fc5ef1d7 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.756648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.757361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.757657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.757940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.758666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.759324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 795/1589] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.760239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.761196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.776786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ef3f-8486-4087-8847-3a7a168d9f17 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.782214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.782961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.783252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.783533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.784364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.787900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 795/1590] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:20 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:20.787900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.788195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.801518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 796/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.801518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.801976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575ee42-aacf-4b0f-a3dd-186588769da2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.806868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.808066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.808912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.809205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.810193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.811813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.812945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea99c10-fbec-41ad-a335-34f7166aae04 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.817743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.818418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.818702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.819140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.819887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.821783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.828579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 796/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.828579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.838915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 797/1593] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.838915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.839613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19947f1b-d591-4ade-84cf-a08e86908a06 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.845783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.846460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.846739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.847012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.847607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4e66a-fa9a-45b3-8424-df25dd5ce77b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.847723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.849599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.852593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.853265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.853567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.853829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.854564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.856426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.868237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 797/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.868237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.868478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 798/1595] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.868478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.883201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fc7f8-4138-4075-ade7-1da8c9ee210b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.887767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.888674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.889192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.889465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91477-25d4-45a9-870b-a23efa78c498 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.889633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.890667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.892964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.896574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.897420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.897874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.898293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.899486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.901652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.904943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 798/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.909291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.915567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 799/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:20.916687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.922117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffac6ed-0ba0-4b2a-b957-047817152b2b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.927703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.927861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ae3d-29e2-4622-9f4d-2e497eb82296 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:20.928848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.929355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.929812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] PATH_INFO: `/v3/projects/3e58d76101894cafbe0f69e7c8d6891e/users/fb4900ec7bcf4ec9ae115e8e81c60de1/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.932659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:20.933663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:20.933973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:20.934269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] PATH_INFO: `/v3/projects/ffec3ea72f144561a4416ae779f476af/users/12e27bf96c3448b2b41cbab55cd5aeaa/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:20.947613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e58d76101894cafbe0f69e7c8d6891e, user_id=fb4900ec7bcf4ec9ae115e8e81c60de1, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.950478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffec3ea72f144561a4416ae779f476af, user_id=12e27bf96c3448b2b41cbab55cd5aeaa, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:20.950647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.954996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:20.966019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 799/1598] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:20 2022] PUT /identity/v3/projects/3e58d76101894cafbe0f69e7c8d6891e/users/fb4900ec7bcf4ec9ae115e8e81c60de1/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:20.966826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:20.977285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 800/1599] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:20 2022] PUT /identity/v3/projects/ffec3ea72f144561a4416ae779f476af/users/12e27bf96c3448b2b41cbab55cd5aeaa/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:20.978109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.052029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd94be-1615-42ce-91a8-35f13cd827c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.052153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a4474-a973-48b1-bbb8-8275841bcba7 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.060876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.061801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.062082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.062363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] PATH_INFO: `/v3/projects/6451d4335ebf46bc95875604279e780e/users/20e3f8325dd24d05a1ce4f083f8afd9b/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.065557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.067979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.068326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.068660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] PATH_INFO: `/v3/projects/f00f2c2d2dea47e4b868ba84f0dbf5ea/users/c8b9ea99092844d59797bbae13a6d362/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.071524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:21.080188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6451d4335ebf46bc95875604279e780e, user_id=20e3f8325dd24d05a1ce4f083f8afd9b, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.083501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.090973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00f2c2d2dea47e4b868ba84f0dbf5ea, user_id=c8b9ea99092844d59797bbae13a6d362, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.095149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.104028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 801/1600] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:20 2022] PUT /identity/v3/projects/6451d4335ebf46bc95875604279e780e/users/20e3f8325dd24d05a1ce4f083f8afd9b/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:21.104028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.125757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 800/1601] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:20 2022] PUT /identity/v3/projects/f00f2c2d2dea47e4b868ba84f0dbf5ea/users/c8b9ea99092844d59797bbae13a6d362/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:21.126609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.137190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:21.186342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dba83-9b5a-4054-856f-56d951a0f893 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.192554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.193485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.193764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.193880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbfaf-4268-487a-aff0-8dd77a4bb9da admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.194287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] PATH_INFO: `/v3/projects/c06b1cb0fcef4cbba01134acdba481be/users/b7a8708943a94915970a0ff89b9f17e3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.199055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c024216-1f9f-4c8d-8cc3-d5b2a3b47bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.200117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c024216-1f9f-4c8d-8cc3-d5b2a3b47bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.200404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c024216-1f9f-4c8d-8cc3-d5b2a3b47bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.200679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c024216-1f9f-4c8d-8cc3-d5b2a3b47bfb admin admin] PATH_INFO: `/v3/projects/2e55dd4580184f158efedbad5dc6e8fb/users/4e9ad2c4909c49a6b3df8ece333e70b5/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.210408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06b1cb0fcef4cbba01134acdba481be, user_id=b7a8708943a94915970a0ff89b9f17e3, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.213133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.217077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c024216-1f9f-4c8d-8cc3-d5b2a3b47bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e55dd4580184f158efedbad5dc6e8fb, user_id=4e9ad2c4909c49a6b3df8ece333e70b5, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.219753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c024216-1f9f-4c8d-8cc3-d5b2a3b47bfb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.229940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 802/1602] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:21 2022] PUT /identity/v3/projects/c06b1cb0fcef4cbba01134acdba481be/users/b7a8708943a94915970a0ff89b9f17e3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:21.231189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.233493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 801/1603] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:21 2022] PUT /identity/v3/projects/2e55dd4580184f158efedbad5dc6e8fb/users/4e9ad2c4909c49a6b3df8ece333e70b5/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:21.234761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bff3c-7702-44b7-a508-6d2ef07ed447 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.235056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bff3c-7702-44b7-a508-6d2ef07ed447 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.235338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bff3c-7702-44b7-a508-6d2ef07ed447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.263386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bd9bff3c-7702-44b7-a508-6d2ef07ed447 None None] MFA Rules not processed for user `12e27bf96c3448b2b41cbab55cd5aeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:21.264787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bd9bff3c-7702-44b7-a508-6d2ef07ed447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:21.309996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6567-36aa-4cea-a5ab-d1aa31690d23 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.343117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 802/1604] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:21.348009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c284cd7-f6af-4f41-8933-09d4b2bba163 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.348308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c284cd7-f6af-4f41-8933-09d4b2bba163 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.351125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c284cd7-f6af-4f41-8933-09d4b2bba163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.380435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc825b2-be17-4289-9fd1-a5debc371be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.380968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc825b2-be17-4289-9fd1-a5debc371be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.381373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc825b2-be17-4289-9fd1-a5debc371be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.381739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc825b2-be17-4289-9fd1-a5debc371be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.382588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc825b2-be17-4289-9fd1-a5debc371be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.384412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc825b2-be17-4289-9fd1-a5debc371be6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.389084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9c284cd7-f6af-4f41-8933-09d4b2bba163 None None] MFA Rules not processed for user `fb4900ec7bcf4ec9ae115e8e81c60de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:21.390416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9c284cd7-f6af-4f41-8933-09d4b2bba163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:21.396512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 803/1605] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:21.397824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46211437-aedc-46e0-ac57-3aae17af30e8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.398112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46211437-aedc-46e0-ac57-3aae17af30e8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.398389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46211437-aedc-46e0-ac57-3aae17af30e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.428195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:21.432414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-46211437-aedc-46e0-ac57-3aae17af30e8 None None] MFA Rules not processed for user `20e3f8325dd24d05a1ce4f083f8afd9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:21.436080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-46211437-aedc-46e0-ac57-3aae17af30e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:21.510488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 803/1606] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:21.515397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.515689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.515989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.539815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:21.549985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 804/1607] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:21.558449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cfadb-4662-4cf5-a5f5-b790979aab0c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.558945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cfadb-4662-4cf5-a5f5-b790979aab0c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.559224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cfadb-4662-4cf5-a5f5-b790979aab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.560999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] MFA Rules not processed for user `c8b9ea99092844d59797bbae13a6d362`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:21.573448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:21.612104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e63cfadb-4662-4cf5-a5f5-b790979aab0c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:21.613381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e63cfadb-4662-4cf5-a5f5-b790979aab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:21.626378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 805/1608] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:21.627418 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.627705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.628011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.647738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:21.648969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:21.657940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 806/1609] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:21.658770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.670191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1b244-7518-49ba-93c2-0536acf9752e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.675103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.675783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.676100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.676379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.677079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.679012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.679470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 804/1610] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:21.680662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.715156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75be6ee-e857-4fd5-a57f-0aa08280dcda None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.721573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c0ac9-223a-4653-b4e6-5fd46f19579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.722951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c0ac9-223a-4653-b4e6-5fd46f19579f admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.722951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c0ac9-223a-4653-b4e6-5fd46f19579f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.723462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c0ac9-223a-4653-b4e6-5fd46f19579f admin admin] PATH_INFO: `/v3/projects/f43a6e910b6b47c09a859ac798bff5cd/users/4b1614d524d24ce486741b599268995a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.728179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 807/1611] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:21.731095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.744853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c0ac9-223a-4653-b4e6-5fd46f19579f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43a6e910b6b47c09a859ac798bff5cd, user_id=4b1614d524d24ce486741b599268995a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.751067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c0ac9-223a-4653-b4e6-5fd46f19579f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.780108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 805/1612] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:21 2022] PUT /identity/v3/projects/f43a6e910b6b47c09a859ac798bff5cd/users/4b1614d524d24ce486741b599268995a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:21.782115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0eb917-88be-4e94-87a1-d810d60f4f8b None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.782472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0eb917-88be-4e94-87a1-d810d60f4f8b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.782837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0eb917-88be-4e94-87a1-d810d60f4f8b None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.785101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 806/1613] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 10:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:21.786014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0eb917-88be-4e94-87a1-d810d60f4f8b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.836613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025fa0b-d980-41d6-b012-467c85bbe821 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.868433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0eb917-88be-4e94-87a1-d810d60f4f8b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:21.873009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.874022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.874370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.874969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.910932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:21.911736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:21.912139 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:21.913841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:21.914766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.917047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.940479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:21.956978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 808/1614] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:21.957781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:21.977171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:21.981539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:21.998861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 807/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:22.010288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.027172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd7d19-1478-41d3-9daa-e8e6fac773e9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.032478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7689c-ad60-45a4-a8b4-f2d1f20a276e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.032780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537c31c-b5aa-49bb-a92a-4d4b1f10097c service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.033397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7689c-ad60-45a4-a8b4-f2d1f20a276e service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.033639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7689c-ad60-45a4-a8b4-f2d1f20a276e service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.033931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7689c-ad60-45a4-a8b4-f2d1f20a276e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.079602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:22.132447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7689c-ad60-45a4-a8b4-f2d1f20a276e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.134363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7689c-ad60-45a4-a8b4-f2d1f20a276e service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.149360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 809/1616] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:22.152295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8939fd-3e58-4a2c-be49-b03778121913 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.152628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8939fd-3e58-4a2c-be49-b03778121913 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.153147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8939fd-3e58-4a2c-be49-b03778121913 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.159448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.161363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.161508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 810/1617] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 10:00:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:22.161658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.162045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.163281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c0aa-b099-4d00-8b5f-a97241b12b0c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.163572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c0aa-b099-4d00-8b5f-a97241b12b0c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.163848 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c0aa-b099-4d00-8b5f-a97241b12b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.164840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.169392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.183385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7bc9c0aa-b099-4d00-8b5f-a97241b12b0c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:22.184781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7bc9c0aa-b099-4d00-8b5f-a97241b12b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:22.196459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 811/1618] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:22.198376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.198630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.199040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.211914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 808/1619] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:21 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:22.213767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.228896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:22.230759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:22.237240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097b843-e33c-4d40-bfe1-ab908d287669 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.244808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5b524-844d-4484-9987-71c18214a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.245900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5b524-844d-4484-9987-71c18214a485 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.252644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5b524-844d-4484-9987-71c18214a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.254470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5b524-844d-4484-9987-71c18214a485 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.256018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5b524-844d-4484-9987-71c18214a485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.260593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5b524-844d-4484-9987-71c18214a485 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.308640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 809/1620] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:22 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:22.308640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.308640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.308640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.353058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:22.354452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:22.356409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 812/1621] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:22.357400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.369941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d2c1-2119-43cf-8e1c-c53bf083b323 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.376052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.376761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.377046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.377323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.378771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.381030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.383553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 810/1622] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:22.384656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.411278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46cb79-41bb-49ee-a9db-a44ad0496129 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.413830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 813/1623] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:22.415011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.416587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.417469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.417926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.418274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.428961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea47e4-3673-4e4d-94ee-fc2752c542bc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.431429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.433392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.435223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.436136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.436778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.437151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.438562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.441747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.446727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 811/1624] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:22.448015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.456289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 814/1625] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:22.457296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.469572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e24d85-9e0c-47c8-a8bb-e67a5f13ba93 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.473352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8ed5-f644-456e-a451-f4410959445c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.477256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.478018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.478326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.478723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.481328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.482336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.482687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.482975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.498617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:22.576595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.577288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.601972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.602286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 815/1626] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:22.605165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.608302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.630561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41af53-0886-4264-b45e-13b60ba416ee service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.640023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 812/1627] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:22.641637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.654478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.656880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8186bb-6a01-46cd-bfb1-9b0aa648c488 service nova] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.658002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.658383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.665642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:22.666233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] PATH_INFO: `/v3/projects/cc5935d2b8294b35a3c374341a9f25ec/users/303abea814554c4db8f7635b1e259652/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.670217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:22.670705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:22.671407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] PATH_INFO: `/v3/projects/df81298ea61a44fc8df32f58b178fdcb/users/19777803437747fabb676dc950009e14/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:22.695269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5935d2b8294b35a3c374341a9f25ec, user_id=303abea814554c4db8f7635b1e259652, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.809278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.809278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 816/1628] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:22 2022] PUT /identity/v3/projects/cc5935d2b8294b35a3c374341a9f25ec/users/303abea814554c4db8f7635b1e259652/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:22.818206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df81298ea61a44fc8df32f58b178fdcb, user_id=19777803437747fabb676dc950009e14, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:22.818206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:22.818512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.818616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 813/1629] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:22 2022] PUT /identity/v3/projects/df81298ea61a44fc8df32f58b178fdcb/users/19777803437747fabb676dc950009e14/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:22.818616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:22.962217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:22.986827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c378a5-cfc2-4238-8830-cd3d347a6777 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:22.999320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:23.000073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.001490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.001774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.004339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:23.008302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:23.011956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c16e57-1431-4a6a-a94c-67d42c481762 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:23.026915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:23.028007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.028443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.029209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.030333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:23.033178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 814/1630] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:23.033827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:23.035151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:23.059979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 817/1631] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:23.068289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e17301-bb5c-4dcf-a969-86efdedc2bf7 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:23.074473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857036f6-76d6-4326-a99d-f42468182b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:23.075559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857036f6-76d6-4326-a99d-f42468182b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.075849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857036f6-76d6-4326-a99d-f42468182b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.076243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857036f6-76d6-4326-a99d-f42468182b3d admin admin] PATH_INFO: `/v3/projects/df81298ea61a44fc8df32f58b178fdcb/users/19777803437747fabb676dc950009e14/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.076566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:23.081668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857036f6-76d6-4326-a99d-f42468182b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df81298ea61a44fc8df32f58b178fdcb, user_id=19777803437747fabb676dc950009e14, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:23.086795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857036f6-76d6-4326-a99d-f42468182b3d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:23.115499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050d3b8-933b-414c-a743-91217e34e51e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:23.136023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 815/1632] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:23 2022] PUT /identity/v3/projects/df81298ea61a44fc8df32f58b178fdcb/users/19777803437747fabb676dc950009e14/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:23.144974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce87358-788e-430e-a861-67ac827e4569 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.144974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce87358-788e-430e-a861-67ac827e4569 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.144974 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce87358-788e-430e-a861-67ac827e4569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.229375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4ce87358-788e-430e-a861-67ac827e4569 None None] MFA Rules not processed for user `19777803437747fabb676dc950009e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:23.232139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4ce87358-788e-430e-a861-67ac827e4569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:23.305291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377a16c-f54b-4ba8-abbc-57c6f0e28614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:23.306201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a16c-f54b-4ba8-abbc-57c6f0e28614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.306649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a16c-f54b-4ba8-abbc-57c6f0e28614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.306777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a16c-f54b-4ba8-abbc-57c6f0e28614 admin admin] PATH_INFO: `/v3/projects/cc5935d2b8294b35a3c374341a9f25ec/users/303abea814554c4db8f7635b1e259652/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.309422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377a16c-f54b-4ba8-abbc-57c6f0e28614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5935d2b8294b35a3c374341a9f25ec, user_id=303abea814554c4db8f7635b1e259652, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:23.311773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377a16c-f54b-4ba8-abbc-57c6f0e28614 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:23.334923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 818/1633] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:23 2022] PUT /identity/v3/projects/cc5935d2b8294b35a3c374341a9f25ec/users/303abea814554c4db8f7635b1e259652/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:23.336326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2af43-6384-4ef2-b79c-ce7d2f8343d2 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.336616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2af43-6384-4ef2-b79c-ce7d2f8343d2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.336896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2af43-6384-4ef2-b79c-ce7d2f8343d2 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.341645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 819/1634] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 10:00:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:23.343028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e9a0e-ef10-4dd0-8804-1bac89c9e2e6 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.343309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e9a0e-ef10-4dd0-8804-1bac89c9e2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.343584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e9a0e-ef10-4dd0-8804-1bac89c9e2e6 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.350114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 820/1635] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 10:00:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:23.351478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d0084-47ca-408e-bd79-691e774aa8e6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.351764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d0084-47ca-408e-bd79-691e774aa8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.352074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d0084-47ca-408e-bd79-691e774aa8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.399843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5c8d0084-47ca-408e-bd79-691e774aa8e6 None None] MFA Rules not processed for user `303abea814554c4db8f7635b1e259652`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:23.402683 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5c8d0084-47ca-408e-bd79-691e774aa8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:23.413237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:23.497611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 816/1636] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:23.498972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.499263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.499543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.555021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 821/1637] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:23.556860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.558315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.559010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.575190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:23.576609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:23.610836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:23.612259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:23.811275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 817/1638] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:23.812299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:23.831525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 822/1639] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:23.832544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:23.909988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d756e8-9f51-40f2-82f4-b755ba01a5e2 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:23.915335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:23.916612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.917290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.918083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:23.925241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:23.973236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ed5a7-5327-48d0-a053-2fa898f1a977 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:23.983366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:23.984359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:23.986155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:23.986548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:24.016125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:24.017778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:24.018332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:24.022828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:24.033141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 818/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:23 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:24.051704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 823/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:23 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:24.113664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:24.129938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d308d98-87a0-4362-83a3-ca3180dc2820 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:24.146049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:24.146995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:24.147287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:24.147565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:24.160279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:24.170609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:24.172431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:24.172741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db454dc4-31f7-489c-a367-44de1f9192ec service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:24.179889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:24.180550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:24.180831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:24.181162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:24.197870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 819/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:24.203998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:24.206197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:24.237421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 824/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:24.628600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:24.640681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88931ed5-3419-4c45-971b-7151f52ed25e service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:24.651071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:24.652324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:24.652658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:24.652943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:24.800699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:24.802990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:24.825774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 820/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:25.125728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:25.158609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5680011-7e50-4228-9309-3053637e7f9c service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:25.166778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bd38f-f6f6-446a-8b8c-95b3c36f247c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:25.167633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bd38f-f6f6-446a-8b8c-95b3c36f247c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:25.168005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bd38f-f6f6-446a-8b8c-95b3c36f247c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:25.168341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bd38f-f6f6-446a-8b8c-95b3c36f247c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:25.211941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:25.332689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:25.404056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bd38f-f6f6-446a-8b8c-95b3c36f247c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:25.408255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bd38f-f6f6-446a-8b8c-95b3c36f247c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:25.445871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 825/1645] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:25.458599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0e79-ab3b-4291-a2a7-5ac854afe9cf service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:25.474774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:25.475795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:25.476172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:25.476551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] PATH_INFO: `/v3/users/c8b9ea99092844d59797bbae13a6d362` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:25.479114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b9ea99092844d59797bbae13a6d362)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:25.481747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:25.550009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:25.589594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b9ea99092844d59797bbae13a6d362'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:25.592359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b9ea99092844d59797bbae13a6d362'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:25.593940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b9ea99092844d59797bbae13a6d362'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:25.617884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b9ea99092844d59797bbae13a6d362'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:25.646463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b9ea99092844d59797bbae13a6d362'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:25.675099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 821/1646] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:25 2022] DELETE /identity/v3/users/c8b9ea99092844d59797bbae13a6d362 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:25.679660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:25.680090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:25.680471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:25.710285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:25.713789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:25.887952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 826/1647] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:26.248051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:26.249553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:26.375357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8486e9-cb2c-47b6-b5fc-8218b65c9d80 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:26.378048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:26.378865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:26.379158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:26.380716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:26.400626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:26.403033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:26.426865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 822/1648] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:28.741059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:28.744722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:28.778084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee8697-746c-4ed5-9e36-b8f8726904b9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:28.805185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:28.806088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:28.806419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:28.806693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] PATH_INFO: `/v3/projects/f00f2c2d2dea47e4b868ba84f0dbf5ea` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:28.810763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00f2c2d2dea47e4b868ba84f0dbf5ea)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:28.811461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:28.824508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:28.827292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:28.864559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa6993-8cf6-4171-976f-a51cb065c492 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:28.891281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:28.892259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:28.892549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:28.892832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] PATH_INFO: `/v3/users/20e3f8325dd24d05a1ce4f083f8afd9b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:28.896402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20e3f8325dd24d05a1ce4f083f8afd9b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:28.898988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:28.944834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00f2c2d2dea47e4b868ba84f0dbf5ea', 'request_id': 'req-918bb77f-6bb4-4001-87df-370f5325bc89'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:28.960585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00f2c2d2dea47e4b868ba84f0dbf5ea', 'request_id': 'req-918bb77f-6bb4-4001-87df-370f5325bc89'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:28.971963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00f2c2d2dea47e4b868ba84f0dbf5ea', 'request_id': 'req-918bb77f-6bb4-4001-87df-370f5325bc89'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:28.981536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20e3f8325dd24d05a1ce4f083f8afd9b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:28.982573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20e3f8325dd24d05a1ce4f083f8afd9b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:28.985596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20e3f8325dd24d05a1ce4f083f8afd9b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:28.999309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] The token cache is being invalidate because project f00f2c2d2dea47e4b868ba84f0dbf5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:28.999603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:29.009150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20e3f8325dd24d05a1ce4f083f8afd9b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:29.010404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-918bb77f-6bb4-4001-87df-370f5325bc89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:29.012107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 827/1649] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:28 2022] DELETE /identity/v3/projects/f00f2c2d2dea47e4b868ba84f0dbf5ea => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:29.016778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:29.017094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:29.017342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:29.036323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6e757639-0dfc-4f83-9d81-99d7ca5fca98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20e3f8325dd24d05a1ce4f083f8afd9b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:29.064007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 823/1650] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:28 2022] DELETE /identity/v3/users/20e3f8325dd24d05a1ce4f083f8afd9b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:29.069193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005e12a-8208-466f-ab56-918a6b0d2350 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:29.069665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005e12a-8208-466f-ab56-918a6b0d2350 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:29.070281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005e12a-8208-466f-ab56-918a6b0d2350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:29.077941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:29.079847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:29.099984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e005e12a-8208-466f-ab56-918a6b0d2350 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:29.102310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e005e12a-8208-466f-ab56-918a6b0d2350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:29.123751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:29.209362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 824/1651] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:29.235123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:29.298720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 828/1652] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:29.308825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:29.308825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:29.309463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:29.353531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:29.357193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:29.382996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 825/1653] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:29.387588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:29.403404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67481a8-78e0-4a21-bd85-89d2a43c8828 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:29.420461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:29.421392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:29.422094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:29.422094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:29.422992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:29.424030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:29.425778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:29.460830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 829/1654] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:29.463762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:29.476182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496e7e8-831c-423d-99b0-6baa6a3cad19 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:29.494448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:29.499421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:29.502268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:29.502551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:29.503299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:29.511233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:29.580426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:29.584229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:29.619797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 826/1655] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:29 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:29.622368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:29.864677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ba02-b5ef-4bd2-9708-1418b6feb811 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:29.955842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a63e-6399-4818-9c24-1ccfeb19e76f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:29.983133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:29.984061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:29.986390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:29.989838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:29.991011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.008454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.089444 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:30.090219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.092474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.092768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.113770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.115239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.120183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 827/1656] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:29 2022] POST /identity/v3/users => generated 287 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:30.128197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.158201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 830/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:30.161967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc5a6-ba13-4c15-b373-d91e3a2d0a7f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:30.181194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:30.181897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.182181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.182797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.183558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.187427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.208784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 828/1658] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:30.212762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.227010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae67aa0-4824-4a87-892a-ca674c949d19 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:30.241746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:30.242751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.243167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.243581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.244466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.251385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.260505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.262768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:30.288799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ef7a-331f-49af-8803-85a0577e3670 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:30.291549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 831/1659] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:30.293134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.306232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:30.312982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.315110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.315396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] PATH_INFO: `/v3/users/12e27bf96c3448b2b41cbab55cd5aeaa` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.316877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e27bf96c3448b2b41cbab55cd5aeaa)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.318840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.329605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908793b3-eeda-4525-aa5f-0b0b0634f38a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:30.354526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50571f-cd09-4b86-a998-8c35f0255ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:30.355631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50571f-cd09-4b86-a998-8c35f0255ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.355826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50571f-cd09-4b86-a998-8c35f0255ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.356189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50571f-cd09-4b86-a998-8c35f0255ec3 admin admin] PATH_INFO: `/v3/projects/2a647b22179d4fa199abdd9b230fdca7/users/592007801ba54382abd736168207ee42/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.409722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50571f-cd09-4b86-a998-8c35f0255ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a647b22179d4fa199abdd9b230fdca7, user_id=592007801ba54382abd736168207ee42, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.417439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50571f-cd09-4b86-a998-8c35f0255ec3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.461193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 832/1660] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:30 2022] PUT /identity/v3/projects/2a647b22179d4fa199abdd9b230fdca7/users/592007801ba54382abd736168207ee42/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:30.472972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.473495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.473905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.476486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e27bf96c3448b2b41cbab55cd5aeaa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:30.479059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e27bf96c3448b2b41cbab55cd5aeaa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:30.482661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e27bf96c3448b2b41cbab55cd5aeaa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:30.501366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e27bf96c3448b2b41cbab55cd5aeaa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:30.537986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:30.563757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-5e130c58-56e5-4a01-9d33-70685a305b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e27bf96c3448b2b41cbab55cd5aeaa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:30.576420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] MFA Rules not processed for user `592007801ba54382abd736168207ee42`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:30.579147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:30.590904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 829/1661] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:30 2022] DELETE /identity/v3/users/12e27bf96c3448b2b41cbab55cd5aeaa => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:30.598320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.598624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.598877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.642222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:30.643468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:30.715368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:30.771044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 830/1662] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:30.780946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.781445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 833/1663] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:30.785052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:30.799421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eb09f-47fb-49f9-8168-0c702b035206 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:30.801147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.802912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:30.806753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:30.807440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:30.808093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:30.808636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:30.810345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:30.815538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:30.874158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 831/1664] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:30 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:30.877933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:30.947213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed385446-d749-4b8f-937c-bd0c78084f4e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.013204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea964f-d278-4f82-bf60-4fd2da247783 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.027484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.028258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.028557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.028842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.029555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.031924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.069910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.073398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.073795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.074074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.077005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 832/1665] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:30 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:00:31.080861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:31.100563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a032f-db8d-4c91-86e3-67f4de78c846 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.106221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.107989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.115744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.117571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.117571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.117794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.118677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.124076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.137011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 834/1666] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:31.155726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 833/1667] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:31.160681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:31.175663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51a526-8374-404e-bdf0-a0deab6d9d88 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.185325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce946bd-d37d-4268-8ddb-e4cf027efaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.187000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce946bd-d37d-4268-8ddb-e4cf027efaeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.187421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce946bd-d37d-4268-8ddb-e4cf027efaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.187842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce946bd-d37d-4268-8ddb-e4cf027efaeb admin admin] PATH_INFO: `/v3/projects/11e01579b77946dca8b7058599c8cd3e/users/ab763d95d66244f3b107fa5c98d19b1b/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.209952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce946bd-d37d-4268-8ddb-e4cf027efaeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e01579b77946dca8b7058599c8cd3e, user_id=ab763d95d66244f3b107fa5c98d19b1b, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.213948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce946bd-d37d-4268-8ddb-e4cf027efaeb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.256631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 835/1668] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:31 2022] PUT /identity/v3/projects/11e01579b77946dca8b7058599c8cd3e/users/ab763d95d66244f3b107fa5c98d19b1b/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:00:31.277437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:31.279529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.279752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.280066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.298243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c061e-9b60-42e9-bb51-82d9375d5ba5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.337398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] MFA Rules not processed for user `a179e34636ed4970be4f0790115f9d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:31.339662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:31.442458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:31.473855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f44e7f-ea01-4340-a82b-6f6bbfffa14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.474927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44e7f-ea01-4340-a82b-6f6bbfffa14a admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.474927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44e7f-ea01-4340-a82b-6f6bbfffa14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.475117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44e7f-ea01-4340-a82b-6f6bbfffa14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.475852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f44e7f-ea01-4340-a82b-6f6bbfffa14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.478302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f44e7f-ea01-4340-a82b-6f6bbfffa14a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.492210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 836/1669] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:31.497297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 834/1670] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:31.505052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:31.506916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.507264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.507611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.517388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa995e-10e8-4ef5-9f8a-da7309417004 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.522260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b73b4-4997-404d-b2f8-62c4be2f4b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.523442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b73b4-4997-404d-b2f8-62c4be2f4b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.523795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b73b4-4997-404d-b2f8-62c4be2f4b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.524180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b73b4-4997-404d-b2f8-62c4be2f4b93 admin admin] PATH_INFO: `/v3/projects/11e01579b77946dca8b7058599c8cd3e/users/ab763d95d66244f3b107fa5c98d19b1b/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.526440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b73b4-4997-404d-b2f8-62c4be2f4b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e01579b77946dca8b7058599c8cd3e, user_id=ab763d95d66244f3b107fa5c98d19b1b, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.529391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b73b4-4997-404d-b2f8-62c4be2f4b93 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.536527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:31.544189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 837/1671] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:31 2022] PUT /identity/v3/projects/11e01579b77946dca8b7058599c8cd3e/users/ab763d95d66244f3b107fa5c98d19b1b/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:31.554668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] MFA Rules not processed for user `a179e34636ed4970be4f0790115f9d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:31.558395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.558692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.558974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.561018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:31.625833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] MFA Rules not processed for user `ab763d95d66244f3b107fa5c98d19b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:31.627237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:31.700104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 835/1672] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:31.707320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:31.708856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:31.722334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 838/1673] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:31.730918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:31.732801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:31.806210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c37d49-6472-4ff8-8685-4399faae5a91 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.813133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.813962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.814992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.814992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.820173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368f2b-0005-4f04-ba5d-c1d7d2780ce5 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:31.827752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920bbbe-ffbe-4768-af37-bb5eb537b8fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:31.828676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920bbbe-ffbe-4768-af37-bb5eb537b8fa service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:31.828966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920bbbe-ffbe-4768-af37-bb5eb537b8fa service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:31.829952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920bbbe-ffbe-4768-af37-bb5eb537b8fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:31.833820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.835787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.840587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920bbbe-ffbe-4768-af37-bb5eb537b8fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:31.842355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920bbbe-ffbe-4768-af37-bb5eb537b8fa service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:31.879476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 839/1674] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:31.881094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 836/1675] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:00:32.580498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:32.789864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b031fbe-2618-434f-9458-d69d2caad7a6 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:32.800630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:32.801942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:32.802357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:32.802768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] PATH_INFO: `/v3/users/592007801ba54382abd736168207ee42` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:32.804586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592007801ba54382abd736168207ee42)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:32.806760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: WARNING py.warnings [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:00:32.806760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: warnings.warn(msg) Jun 30 10:00:32.806760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: Jun 30 10:00:32.808818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:32.839274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89f4f9-f631-4654-b1e0-69c2edc6fbd8 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:32.839788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89f4f9-f631-4654-b1e0-69c2edc6fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:32.839924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89f4f9-f631-4654-b1e0-69c2edc6fbd8 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:32.842119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 837/1676] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 10:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:32.852229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:32.852520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:32.852807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:32.937575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:32.938986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.001253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:33.039592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 838/1677] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:00:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:33.040991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:33.042074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.057119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:33.065893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1ba23-e9ff-4381-95bb-1327e105444b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:33.083742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592007801ba54382abd736168207ee42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.084151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:33.085234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:33.085607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592007801ba54382abd736168207ee42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.087085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:33.087580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] PATH_INFO: `/v3/projects/ffec3ea72f144561a4416ae779f476af` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:33.089611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffec3ea72f144561a4416ae779f476af)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:33.091727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: WARNING py.warnings [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:00:33.091727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: warnings.warn(msg) Jun 30 10:00:33.091727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: Jun 30 10:00:33.095234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:33.116947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592007801ba54382abd736168207ee42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.117910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592007801ba54382abd736168207ee42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.152914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592007801ba54382abd736168207ee42'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.192962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 840/1678] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:32 2022] DELETE /identity/v3/users/592007801ba54382abd736168207ee42 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:33.192962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:33.193974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.210422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffec3ea72f144561a4416ae779f476af', 'request_id': 'req-ed8485e9-38c5-466b-a2e7-070c8802ebc2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.219281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffec3ea72f144561a4416ae779f476af', 'request_id': 'req-ed8485e9-38c5-466b-a2e7-070c8802ebc2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.234575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffec3ea72f144561a4416ae779f476af', 'request_id': 'req-ed8485e9-38c5-466b-a2e7-070c8802ebc2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.267399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] The token cache is being invalidate because project ffec3ea72f144561a4416ae779f476af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:33.268334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.269934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ed8485e9-38c5-466b-a2e7-070c8802ebc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.281517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 839/1679] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:33 2022] DELETE /identity/v3/projects/ffec3ea72f144561a4416ae779f476af => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:33.298837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:33.299348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:33.299980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:33.320519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd435af3-a382-428f-9334-3ad4b84b9234 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:33.325155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:33.326125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:33.326505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:33.326870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:33.328302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:33.329506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.330778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.345399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:33.346802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:33.362254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 841/1680] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:33.371125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:33.373368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.412464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 840/1681] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:33.435417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:33.438090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:33.467859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf680-74a5-45fe-815d-2be314fcf26e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:33.487767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc52a0-0573-4ace-8041-8b1e8f219276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:33.488539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc52a0-0573-4ace-8041-8b1e8f219276 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:33.488889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc52a0-0573-4ace-8041-8b1e8f219276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:33.489231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc52a0-0573-4ace-8041-8b1e8f219276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:33.500672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef454e-1667-4a56-a950-a69ee1846579 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:33.508175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:33.509079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:33.509351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:33.509628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] PATH_INFO: `/v3/users/303abea814554c4db8f7635b1e259652` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:33.510987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303abea814554c4db8f7635b1e259652)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:33.513038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:33.514147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc52a0-0573-4ace-8041-8b1e8f219276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:33.518466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc52a0-0573-4ace-8041-8b1e8f219276 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:33.554511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 841/1682] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:33.592604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303abea814554c4db8f7635b1e259652'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.593701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303abea814554c4db8f7635b1e259652'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.596740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:33.613940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303abea814554c4db8f7635b1e259652'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.614739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303abea814554c4db8f7635b1e259652'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.641409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8c4d9299-3859-4f24-97ac-6211a0f82dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303abea814554c4db8f7635b1e259652'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:33.665556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 842/1683] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:33 2022] DELETE /identity/v3/users/303abea814554c4db8f7635b1e259652 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:34.198641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c132b-1ae0-41a8-81e5-a2f4e71b2708 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:34.199557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c132b-1ae0-41a8-81e5-a2f4e71b2708 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:34.200316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c132b-1ae0-41a8-81e5-a2f4e71b2708 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:34.201539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 842/1684] 10.222.0.16 () {54 vars in 960 bytes} [Thu Jun 30 10:00:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:34.208254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:34.208894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:34.209426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:34.254943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:34.257614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:34.385057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785584-0893-4082-a563-dd8d06015022 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:34.385493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785584-0893-4082-a563-dd8d06015022 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:34.386083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785584-0893-4082-a563-dd8d06015022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:34.433104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-7c785584-0893-4082-a563-dd8d06015022 None None] MFA Rules not processed for user `a179e34636ed4970be4f0790115f9d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:34.434726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7c785584-0893-4082-a563-dd8d06015022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:34.514524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 843/1685] 10.222.0.16 () {58 vars in 1057 bytes} [Thu Jun 30 10:00:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:34.516558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:34.539588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eed8b6-0598-4a2a-bfc1-d2512e9ca976 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:34.553933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:34.554926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:34.555240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:34.555982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:34.573400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:34.594967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 843/1686] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:34.595908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c785584-0893-4082-a563-dd8d06015022 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:34.597307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7c785584-0893-4082-a563-dd8d06015022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:34.616879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:34.619167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:34.645525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:34.679564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 844/1687] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:34.688328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:34.692122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:34.797749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c785584-0893-4082-a563-dd8d06015022 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:34.822802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:34.823779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:34.824101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:34.824396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] PATH_INFO: `/v3/projects/2a647b22179d4fa199abdd9b230fdca7` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:34.833350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a647b22179d4fa199abdd9b230fdca7)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:34.834912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:34.924741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149b9cd-4023-48d1-941c-17be62100204 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:34.961793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a647b22179d4fa199abdd9b230fdca7', 'request_id': 'req-a95ece44-85e2-44fa-93d5-2ec4f6c87077'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:34.976535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a647b22179d4fa199abdd9b230fdca7', 'request_id': 'req-a95ece44-85e2-44fa-93d5-2ec4f6c87077'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.000897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a647b22179d4fa199abdd9b230fdca7', 'request_id': 'req-a95ece44-85e2-44fa-93d5-2ec4f6c87077'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.036169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] The token cache is being invalidate because project 2a647b22179d4fa199abdd9b230fdca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:35.037985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.039592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.053936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 844/1688] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:34 2022] DELETE /identity/v3/projects/2a647b22179d4fa199abdd9b230fdca7 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:35.061005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:35.062942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:35.086695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:35.111705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:35.112696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:35.112979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:35.136589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] PATH_INFO: `/v3/projects/cc5935d2b8294b35a3c374341a9f25ec` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:35.141716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc5935d2b8294b35a3c374341a9f25ec)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:35.146731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:35.262237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ece44-85e2-44fa-93d5-2ec4f6c87077 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:35.280558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:35.281467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:35.281747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:35.282020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] PATH_INFO: `/v3/users/ab763d95d66244f3b107fa5c98d19b1b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:35.285850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab763d95d66244f3b107fa5c98d19b1b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:35.290371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:35.332241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc5935d2b8294b35a3c374341a9f25ec', 'request_id': 'req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.348122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc5935d2b8294b35a3c374341a9f25ec', 'request_id': 'req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.366503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc5935d2b8294b35a3c374341a9f25ec', 'request_id': 'req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.400953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] The token cache is being invalidate because project cc5935d2b8294b35a3c374341a9f25ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:35.401147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.405222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-30b5e4e2-d6f2-43ca-9a45-79631dc045d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.407711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 845/1689] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:34 2022] DELETE /identity/v3/projects/cc5935d2b8294b35a3c374341a9f25ec => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:35.415847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75d54-e305-4757-bea1-80b589c003fa None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:35.416162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75d54-e305-4757-bea1-80b589c003fa None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:35.416586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75d54-e305-4757-bea1-80b589c003fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:35.441540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab763d95d66244f3b107fa5c98d19b1b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.443263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab763d95d66244f3b107fa5c98d19b1b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.444973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab763d95d66244f3b107fa5c98d19b1b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.452447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:35.465907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab763d95d66244f3b107fa5c98d19b1b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.504663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab763d95d66244f3b107fa5c98d19b1b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:35.539485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 845/1690] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:35 2022] DELETE /identity/v3/users/ab763d95d66244f3b107fa5c98d19b1b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:35.678206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:35.679964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:35.721396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-09b75d54-e305-4757-bea1-80b589c003fa None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:35.723252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-09b75d54-e305-4757-bea1-80b589c003fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:35.887629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 846/1691] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:35.898469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:35.898792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:35.899014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:35.946209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:35.947804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:35.968896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 847/1692] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:35.974307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.005400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20458-1c05-4d3b-8f56-549756dab56e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.028118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.029771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.031611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.032064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a7a38-29b6-4856-9f42-b8e32ea32345 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.037743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] PATH_INFO: `/v3/projects/6451d4335ebf46bc95875604279e780e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.040624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6451d4335ebf46bc95875604279e780e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.047115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.060513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.078172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.078970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.079392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.080816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:36.081313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.083520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.124431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 848/1693] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:36.127791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.170915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4cb8c-e06b-45a7-8bd0-375e2df08871 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.180749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.185364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.185791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.186508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.187419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.191298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.297749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6451d4335ebf46bc95875604279e780e', 'request_id': 'req-ec4fb354-8d87-4951-8668-0b145bced2f6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:36.328070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6451d4335ebf46bc95875604279e780e', 'request_id': 'req-ec4fb354-8d87-4951-8668-0b145bced2f6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:36.331199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 849/1694] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:36 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:00:36.333699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.356912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6451d4335ebf46bc95875604279e780e', 'request_id': 'req-ec4fb354-8d87-4951-8668-0b145bced2f6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:36.375820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] The token cache is being invalidate because project 6451d4335ebf46bc95875604279e780e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:36.377547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:36.382626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ec4fb354-8d87-4951-8668-0b145bced2f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:36.385894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 846/1695] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:35 2022] DELETE /identity/v3/projects/6451d4335ebf46bc95875604279e780e => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:00:36.392042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739a241-f406-4efb-a0bd-44ffa688cc71 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.392328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739a241-f406-4efb-a0bd-44ffa688cc71 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.392603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739a241-f406-4efb-a0bd-44ffa688cc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.464183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0739a241-f406-4efb-a0bd-44ffa688cc71 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:36.470302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0739a241-f406-4efb-a0bd-44ffa688cc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:36.503770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:36.546361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c7daf-a494-4c68-84b4-172324cb57b5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.561918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.562881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.563302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.563712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.564618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.571087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.628917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 850/1696] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:36 2022] POST /identity/v3/users => generated 291 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:36.632829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.638688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:36.660051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:36.667321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591c4dd-5bd9-4cc2-a753-f11c86b30902 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.679250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.679963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.680254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.680530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.681271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.691767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.701801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 847/1697] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:36.706978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.707586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.708180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.713783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 851/1698] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:36.716210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.742433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb09a-14a9-4d8c-acf7-9a097a0d789b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.757676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:36.764459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.765152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.765436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.765712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.766350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:36.766639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.769338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.793756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 848/1699] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:36.800009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.800140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 852/1700] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:36.802532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.820567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd60cd-570a-4719-bf7a-3d3621076cf8 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.829557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebedf9-e3d4-4139-b299-6530fa828a73 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:36.832757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.835357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.836046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.836624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.838329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:36.840797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.843742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.849482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea4634-275c-4e0b-872a-781a8135db54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:36.852750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4634-275c-4e0b-872a-781a8135db54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.853226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4634-275c-4e0b-872a-781a8135db54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.853992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4634-275c-4e0b-872a-781a8135db54 admin admin] PATH_INFO: `/v3/projects/429be2da119040bca09734b9ece09b34/users/bd936e8a8ee345c981c2f3a0b798ce3a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.890379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea4634-275c-4e0b-872a-781a8135db54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=429be2da119040bca09734b9ece09b34, user_id=bd936e8a8ee345c981c2f3a0b798ce3a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:36.893335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea4634-275c-4e0b-872a-781a8135db54 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:36.950055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 853/1701] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:36 2022] PUT /identity/v3/projects/429be2da119040bca09734b9ece09b34/users/bd936e8a8ee345c981c2f3a0b798ce3a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:36.956034 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:36.956514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:36.957093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 849/1702] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:36.957192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:36.960440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:36.994431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bd003-083c-41d8-92b0-e6d9d1841c69 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:37.121334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] MFA Rules not processed for user `bd936e8a8ee345c981c2f3a0b798ce3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:37.129559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:37.174175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:37.174874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:37.175154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:37.175639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:37.176159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:37.191131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:37.281941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 850/1703] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:36 2022] POST /identity/v3/projects => generated 403 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:37.283234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:37.288742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:37.455154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d838e-4a10-402e-ae72-da48a0c455fe admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:37.464117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:37.465498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 854/1704] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:37.465841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:37.466333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:37.467131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] PATH_INFO: `/v3/projects/11e01579b77946dca8b7058599c8cd3e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:37.467405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:37.469115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e01579b77946dca8b7058599c8cd3e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:37.473236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:37.508185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b509e-acd5-4b90-8eff-88a6ef50778d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:37.528677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:37.529365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:37.529654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:37.530082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:37.530794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:37.535011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:37.640405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 855/1705] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:37 2022] POST /identity/v3/users => generated 302 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:37.643863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:37.657684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e01579b77946dca8b7058599c8cd3e', 'request_id': 'req-2a76f133-76e3-4ae8-8dc1-4f801a880216'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:37.671306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e01579b77946dca8b7058599c8cd3e', 'request_id': 'req-2a76f133-76e3-4ae8-8dc1-4f801a880216'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:37.693114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147d26-dba1-4c61-9609-7591b79fa220 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:37.697526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e01579b77946dca8b7058599c8cd3e', 'request_id': 'req-2a76f133-76e3-4ae8-8dc1-4f801a880216'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:37.700075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:37.772063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] The token cache is being invalidate because project 11e01579b77946dca8b7058599c8cd3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:37.772365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:37.776808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:37.783714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 851/1706] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:37 2022] DELETE /identity/v3/projects/11e01579b77946dca8b7058599c8cd3e => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:37.784095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:37.792408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:37.928811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b1bb-3c5a-4c7b-a40d-df365889890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:37.929490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b1bb-3c5a-4c7b-a40d-df365889890a admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:37.929770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b1bb-3c5a-4c7b-a40d-df365889890a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:37.930042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b1bb-3c5a-4c7b-a40d-df365889890a admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:37.930741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b1bb-3c5a-4c7b-a40d-df365889890a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:37.932763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b1bb-3c5a-4c7b-a40d-df365889890a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:37.941496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:37.991368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f133-76e3-4ae8-8dc1-4f801a880216 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:37.997811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:37.998579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:37.999123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:37.999582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.000523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.001843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 856/1707] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:37 2022] POST /identity/v3/projects => generated 379 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:38.002840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.003429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.003645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.010946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.027785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 852/1708] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:38.029639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:38.032378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:38.047099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dae142-2539-4b80-9597-eb5fd27ceddb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:38.060314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:38.064823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:38.219150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147f855-dd04-49dc-9c19-990c581e1277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:38.222801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147f855-dd04-49dc-9c19-990c581e1277 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.223110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147f855-dd04-49dc-9c19-990c581e1277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.223405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147f855-dd04-49dc-9c19-990c581e1277 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.228048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147f855-dd04-49dc-9c19-990c581e1277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.238996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147f855-dd04-49dc-9c19-990c581e1277 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.246351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 857/1709] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:38.247598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:38.272120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090a5c3-67e3-4517-8f88-3cc3d8876fdb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:38.286044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:38.288342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.288635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.288914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.291661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.295084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.323450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 853/1710] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:38 2022] POST /identity/v3/users => generated 296 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:38.329233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.329972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.331124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.334469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 858/1711] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:38.336125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:38.370512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131895c9-30d5-4d44-8c14-ac18f9065618 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:38.396525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:38.397219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.404851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.405244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.408587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:38.408983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.416223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:38.419294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.459561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 854/1712] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:38.460583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:38.483690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 859/1713] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:38.487290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:38.496474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33506b1a-e81b-485c-ac2e-6650dc0e0f9b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:38.514960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:38.521203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.521520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.523122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] PATH_INFO: `/v3/projects/3ef8281ebf864fbbb863e99fa091ce0e/users/23bab66158b443d6b530a20323313483/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.560891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a592f-56b9-40de-b8b2-08c3a23b81a9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:38.580624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef8281ebf864fbbb863e99fa091ce0e, user_id=23bab66158b443d6b530a20323313483, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.593723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:38.594002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.596819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.598051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.599137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.602039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:38.602387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.612456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd9b44-ad4e-4d12-b846-463181de37ce admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.644666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 855/1714] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:38 2022] PUT /identity/v3/projects/3ef8281ebf864fbbb863e99fa091ce0e/users/23bab66158b443d6b530a20323313483/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:38.645695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:38.667339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 860/1715] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:38.671235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.671832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.673355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.726486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] MFA Rules not processed for user `b7a8708943a94915970a0ff89b9f17e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:38.728177 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:38.792269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:38.815981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd73fab-49bf-4016-a87d-167e09fcbdab admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:38.822458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306571-6614-449c-971e-b0d360685e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:38.824786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306571-6614-449c-971e-b0d360685e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.825335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306571-6614-449c-971e-b0d360685e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.825759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306571-6614-449c-971e-b0d360685e3d admin admin] PATH_INFO: `/v3/projects/970f9b23187a41d7ae3054d6a3abe6df/users/c59c359a96c64e9ca3b55c6467a0c611/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.860316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63306571-6614-449c-971e-b0d360685e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=970f9b23187a41d7ae3054d6a3abe6df, user_id=c59c359a96c64e9ca3b55c6467a0c611, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:38.868149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63306571-6614-449c-971e-b0d360685e3d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:38.949161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 856/1716] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:38 2022] PUT /identity/v3/projects/970f9b23187a41d7ae3054d6a3abe6df/users/c59c359a96c64e9ca3b55c6467a0c611/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:38.953027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664921-ef89-491c-8135-c3c84de93cd6 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:38.953201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664921-ef89-491c-8135-c3c84de93cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:38.953481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664921-ef89-491c-8135-c3c84de93cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:38.991492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 861/1717] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:38.991492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.008634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe268-dad6-4286-94fa-664921a1b675 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:39.027254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d9664921-ef89-491c-8135-c3c84de93cd6 None None] MFA Rules not processed for user `23bab66158b443d6b530a20323313483`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:39.029702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d9664921-ef89-491c-8135-c3c84de93cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:39.085293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:39.118877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:39.120012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.120459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.121175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.122262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:39.130912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:39.182750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 862/1718] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:38 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:39.184176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.216732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 857/1719] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:39.221132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.221435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.221721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.297119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:39.298395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:39.306775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc75b4-5b00-461c-bae6-6741ed3410cd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:39.319355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b59b7a-92b2-41d6-8346-9e8152f96dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:39.320076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b59b7a-92b2-41d6-8346-9e8152f96dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.320358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b59b7a-92b2-41d6-8346-9e8152f96dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.320637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b59b7a-92b2-41d6-8346-9e8152f96dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.321689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b59b7a-92b2-41d6-8346-9e8152f96dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:39.329100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b59b7a-92b2-41d6-8346-9e8152f96dbc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:39.361984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 863/1720] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:39.362273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.362703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.366053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.419766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] MFA Rules not processed for user `b7a8708943a94915970a0ff89b9f17e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:39.421356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:39.495919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 858/1721] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:00:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:39.508147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.528816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 864/1722] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:39.528816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.558483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f90f7-ebe9-4727-b2f0-38dce37a6479 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:39.563965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad4e44-badf-4b15-993b-63d37654aec5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:39.578562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:39.581401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.582679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.583182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:39.583783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.584596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:39.585699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.588179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.588474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.590424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:39.594387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:39.597988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:39.706177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 859/1723] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:39 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:39.706177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.748825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bbf2e-9c5c-4aca-924f-761ea151d759 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:39.768685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 865/1724] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:39 2022] POST /identity/v3/projects => generated 405 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:39.768685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.770358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:39.771893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:39.772186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:39.772480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] PATH_INFO: `/v3/projects/970f9b23187a41d7ae3054d6a3abe6df/users/c59c359a96c64e9ca3b55c6467a0c611/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:39.775317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:39.777112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] RBAC: Authorizing `identity:create_grant(project_id=970f9b23187a41d7ae3054d6a3abe6df, user_id=c59c359a96c64e9ca3b55c6467a0c611, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:39.780071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:39.823934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 860/1725] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:39 2022] PUT /identity/v3/projects/970f9b23187a41d7ae3054d6a3abe6df/users/c59c359a96c64e9ca3b55c6467a0c611/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:39.826669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:39.915124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:39.940499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:39.996074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5fcf2-47a9-4d59-836b-45e1d0a4f43c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.005207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963b998-7ca6-4c04-830d-3d8b31fc01be admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.025424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.026289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.026668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.026948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.043421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.048684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.092807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 861/1726] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:40.095743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.135300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbc0a1-18d8-40c5-accd-a8f74d783b42 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.150879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f058-7648-455f-bf82-e87bbfe4a0f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.152447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f058-7648-455f-bf82-e87bbfe4a0f0 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.153875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f058-7648-455f-bf82-e87bbfe4a0f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.154257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f058-7648-455f-bf82-e87bbfe4a0f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.157176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.157919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.158200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.159081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.160963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.164876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.240346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 862/1727] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:40 2022] POST /identity/v3/users => generated 309 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:40.241444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.244766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:40.352652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f058-7648-455f-bf82-e87bbfe4a0f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.356620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f058-7648-455f-bf82-e87bbfe4a0f0 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.381772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 866/1728] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:40.400410 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.401829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.403604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.473000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] MFA Rules not processed for user `c59c359a96c64e9ca3b55c6467a0c611`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:40.474478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:40.600831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 867/1729] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:40.602116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.614030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae562810-7647-46d2-935f-1f5ba78e0ac3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.619244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.620068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.620352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.620718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.621485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.623081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019cd1-f8cd-49d8-bc24-e8d034b5167a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.623569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.624949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:40.638505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 868/1730] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:40.639822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.651212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.652868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.653676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076f4d4-8f8b-4b82-a275-c488e72fd3d0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.654012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.654359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.658931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.659601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.659928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.660182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.661012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.662619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.675452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 869/1731] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:40.676382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.680281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:40.794057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86745ffd-417c-4c3e-824d-d79d723ac2ff admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.797844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.799681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.800449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.801508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.802245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.802805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.816354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 863/1732] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:00:40.817367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.824641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.826056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:40.844803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b965e8-085f-4761-a817-279ff4ae2d6c service swift] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.874026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71311e38-3c28-4227-8878-d03c166435b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.877356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71311e38-3c28-4227-8878-d03c166435b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.885286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71311e38-3c28-4227-8878-d03c166435b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.887003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71311e38-3c28-4227-8878-d03c166435b7 admin admin] PATH_INFO: `/v3/projects/037e84f88f8848cca8e1befb934f5744/users/ae716322b814408c881eff4058a54ded/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.896340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 870/1733] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:40.900722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:40.942716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a7f4-6f8b-4485-bc44-8f6cc36b0dd8 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:40.949663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:40.950912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:40.951199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:40.962727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] PATH_INFO: `/v3/projects/849aae572bae4d358d22f7012c69b3b8/users/17cfd847b97f4868922bc6266ccbd8ff/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:40.972154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71311e38-3c28-4227-8878-d03c166435b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=037e84f88f8848cca8e1befb934f5744, user_id=ae716322b814408c881eff4058a54ded, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:40.998407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71311e38-3c28-4227-8878-d03c166435b7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:41.020879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=849aae572bae4d358d22f7012c69b3b8, user_id=17cfd847b97f4868922bc6266ccbd8ff, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:41.027533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:41.040369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 864/1734] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:40 2022] PUT /identity/v3/projects/037e84f88f8848cca8e1befb934f5744/users/ae716322b814408c881eff4058a54ded/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:41.048229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:41.048722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:41.049256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:41.114327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 871/1735] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:40 2022] PUT /identity/v3/projects/849aae572bae4d358d22f7012c69b3b8/users/17cfd847b97f4868922bc6266ccbd8ff/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:41.118831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:41.135160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] MFA Rules not processed for user `ae716322b814408c881eff4058a54ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:41.137227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:41.178788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:41.263578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 865/1736] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:41.265423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:41.394459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7f69e-29e9-4d3d-8148-6cb6d2aaa1df admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:41.414486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:41.419733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:41.422112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:41.422651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:41.425401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:41.437548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:41.457571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1fe79-ea8e-4fab-8879-ee74e70ba4d7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:41.476949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 872/1737] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:00:41.489707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:41.489930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:41.490672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:41.491074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:41.491456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:41.492256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:41.494489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:41.522166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91238e9-a6ec-414e-9550-653eaff15876 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:41.541285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269349e-7ff8-4bad-801c-4b89017484ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:41.545529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269349e-7ff8-4bad-801c-4b89017484ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:41.546937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269349e-7ff8-4bad-801c-4b89017484ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:41.548125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269349e-7ff8-4bad-801c-4b89017484ae admin admin] PATH_INFO: `/v3/projects/849aae572bae4d358d22f7012c69b3b8/users/17cfd847b97f4868922bc6266ccbd8ff/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:41.553649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269349e-7ff8-4bad-801c-4b89017484ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=849aae572bae4d358d22f7012c69b3b8, user_id=17cfd847b97f4868922bc6266ccbd8ff, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:41.558979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269349e-7ff8-4bad-801c-4b89017484ae admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:41.586066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 873/1738] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:41 2022] PUT /identity/v3/projects/849aae572bae4d358d22f7012c69b3b8/users/17cfd847b97f4868922bc6266ccbd8ff/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:41.601012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:41.601622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:41.604414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:41.629157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 866/1739] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:41 2022] POST /identity/v3/projects => generated 363 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:00:41.631035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:41.723298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] MFA Rules not processed for user `17cfd847b97f4868922bc6266ccbd8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:41.727241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:41.900356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:42.057694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b20041-d38f-491b-bd81-77a8d5b90a4b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:42.077854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 874/1740] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:42.079020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:42.095381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09993b09-e6bd-4165-b478-2bb75be4a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:42.105714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09993b09-e6bd-4165-b478-2bb75be4a8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.106001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09993b09-e6bd-4165-b478-2bb75be4a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.114314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09993b09-e6bd-4165-b478-2bb75be4a8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.115561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09993b09-e6bd-4165-b478-2bb75be4a8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:42.132664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09993b09-e6bd-4165-b478-2bb75be4a8c4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:42.218756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 867/1741] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:41 2022] POST /identity/v3/users => generated 288 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:42.220521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2447a-f0d4-4011-947c-7565efd5c078 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.220816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2447a-f0d4-4011-947c-7565efd5c078 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.221082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2447a-f0d4-4011-947c-7565efd5c078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.320544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2345a9c-9ee3-42ce-8481-3c79c37548ce None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:42.332415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c59c359a96c64e9ca3b55c6467a0c611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '970f9b23187a41d7ae3054d6a3abe6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:42.333176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.333466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.333749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.334504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:42.334814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:42.337158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8ea2447a-f0d4-4011-947c-7565efd5c078 None None] MFA Rules not processed for user `4e9ad2c4909c49a6b3df8ece333e70b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:42.338266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8ea2447a-f0d4-4011-947c-7565efd5c078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:42.340616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:42.408236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 875/1742] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:42.410525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:42.418756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:42.449997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:42.618237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 868/1743] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:42.621059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a2a95-6898-45e5-931a-60f287ac1b55 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.622769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a2a95-6898-45e5-931a-60f287ac1b55 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.623055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a2a95-6898-45e5-931a-60f287ac1b55 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.624683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 869/1744] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 10:00:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:42.625498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a2a95-6898-45e5-931a-60f287ac1b55 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:42.658828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ca098-0180-4ddf-a15a-4e9167d42a6c tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:42.668501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ef38e-8f6c-4bc1-9907-98350811e327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:42.669592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ef38e-8f6c-4bc1-9907-98350811e327 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.670085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ef38e-8f6c-4bc1-9907-98350811e327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.670641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ef38e-8f6c-4bc1-9907-98350811e327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.674282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a2a95-6898-45e5-931a-60f287ac1b55 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:42.684091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:42.685109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.685486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.685786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.686599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:42.689112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:42.706393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ef38e-8f6c-4bc1-9907-98350811e327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:42.708270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ef38e-8f6c-4bc1-9907-98350811e327 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:42.709304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 870/1745] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:42.711818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:42.736825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 876/1746] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:42 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:42.738730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.741452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dc11a-5375-4e99-b65c-ef5323e76f64 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:42.741769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.742254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.761851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c59c359a96c64e9ca3b55c6467a0c611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '970f9b23187a41d7ae3054d6a3abe6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:42.763395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:42.764030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:42.764653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:42.765516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:42.768819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:42.807922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:42.816547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:42.840559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 871/1747] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:42 2022] POST /identity/v3/projects => generated 353 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:42.841597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:42.843565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:43.032448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e958cd6-6d4e-4891-b360-6a174e2e8267 tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:43.039914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 877/1748] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:43.041202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:43.052134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:43.054638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:43.054937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:43.055412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:43.082225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fecb7e-0a5f-48ed-b6ae-e6168bfd2093 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:43.121478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:43.233956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:43.235052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:43.235469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:43.235852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] PATH_INFO: `/v3/projects/f403c78428af4f5491ee0bcaa282f337/users/dcd34aa8318b41c8be7adbbcc84c61fa/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:43.281225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f403c78428af4f5491ee0bcaa282f337, user_id=dcd34aa8318b41c8be7adbbcc84c61fa, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:43.298716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:43.303720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:43.305618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:43.342762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 872/1749] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:43.345084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:43.355332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 878/1750] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:43 2022] PUT /identity/v3/projects/f403c78428af4f5491ee0bcaa282f337/users/dcd34aa8318b41c8be7adbbcc84c61fa/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:43.368836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:43.463318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:43.609560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765a922-ef8d-4488-afa9-54a794e657b9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:43.612588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5a49-52ae-47a3-8c94-f06942c3392f service nova] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:43.648491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:43.649905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:43.651804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:43.655931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] PATH_INFO: `/v3/users/23bab66158b443d6b530a20323313483` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:43.661897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bab66158b443d6b530a20323313483)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:43.667103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:43.882993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bab66158b443d6b530a20323313483'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:43.892393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bab66158b443d6b530a20323313483'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:43.911753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c59c359a96c64e9ca3b55c6467a0c611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '970f9b23187a41d7ae3054d6a3abe6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:43.913173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:43.913814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:43.914047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:43.915058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:43.915434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:43.927269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bab66158b443d6b530a20323313483'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:43.927480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:43.928355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bab66158b443d6b530a20323313483'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:43.972337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bab66158b443d6b530a20323313483'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:43.996057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 873/1751] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 652 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:44.004699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:44.007938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:44.034373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 879/1752] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:43 2022] DELETE /identity/v3/users/23bab66158b443d6b530a20323313483 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:44.034373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:44.040292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:44.074010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6e79-1afb-43bc-a089-b3359d795fcf tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:44.106766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:44.299723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a13f-fefe-4aef-b1fa-d12e79f0b14b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:44.313694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fe8fe-8306-44df-b68a-dbad383633d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:44.315593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fe8fe-8306-44df-b68a-dbad383633d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:44.316420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fe8fe-8306-44df-b68a-dbad383633d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:44.317156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fe8fe-8306-44df-b68a-dbad383633d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:44.318221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fe8fe-8306-44df-b68a-dbad383633d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:44.320176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fe8fe-8306-44df-b68a-dbad383633d5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:44.324659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:44.325400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:44.325686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:44.325971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:44.333041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:44.360613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 874/1753] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:44.360901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:44.361133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:44.363168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab22c2-e0a4-47a6-9b51-d93a7879fe5e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:44.363915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab22c2-e0a4-47a6-9b51-d93a7879fe5e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:44.364602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab22c2-e0a4-47a6-9b51-d93a7879fe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:44.391437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-cdab22c2-e0a4-47a6-9b51-d93a7879fe5e None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:44.393309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cdab22c2-e0a4-47a6-9b51-d93a7879fe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:44.404437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 875/1754] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 30 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:44.406864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:44.407395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:44.407905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:44.417131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 880/1755] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:44.419036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:44.429832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:44.437481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:44.438966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:44.452245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 876/1756] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:44.452830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:44.468651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27c281-ad71-41de-b00f-2c876054520a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:44.479021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:44.480529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:44.480870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:44.481337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] PATH_INFO: `/v3/projects/f403c78428af4f5491ee0bcaa282f337/users/dcd34aa8318b41c8be7adbbcc84c61fa/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:44.487155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f403c78428af4f5491ee0bcaa282f337, user_id=dcd34aa8318b41c8be7adbbcc84c61fa, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:44.488151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:44.540631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 877/1757] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:44 2022] PUT /identity/v3/projects/f403c78428af4f5491ee0bcaa282f337/users/dcd34aa8318b41c8be7adbbcc84c61fa/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:44.542349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:44.572789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8015bd2-80b5-4950-97b9-c66b09146d4a service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:44.728606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c59c359a96c64e9ca3b55c6467a0c611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '970f9b23187a41d7ae3054d6a3abe6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:44.729592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:44.730084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:44.730264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] PATH_INFO: `/v3/projects/9ec5a69b298b47ffa5130ed093649e66` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:44.734200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9ec5a69b298b47ffa5130ed093649e66)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:44.736312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:44.749066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:44.811296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946ed97-2839-4543-a52c-11e6dc833c46 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:44.865867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec5a69b298b47ffa5130ed093649e66', 'request_id': 'req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:44.883341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec5a69b298b47ffa5130ed093649e66', 'request_id': 'req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:44.892370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec5a69b298b47ffa5130ed093649e66', 'request_id': 'req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:44.915865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:44.929455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] The token cache is being invalidate because project 9ec5a69b298b47ffa5130ed093649e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:44.930201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:44.931850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:44.935539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 881/1758] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:44 2022] DELETE /identity/v3/projects/9ec5a69b298b47ffa5130ed093649e66 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:44.936696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:44.940961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:45.103026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88167f-0d5f-4bc0-9c4b-c9c720af841b tempest-VolumeQuotaClassesTest-288089449 tempest-VolumeQuotaClassesTest-288089449-project-admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:45.122588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:45.123262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:45.123786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:45.123786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:45.131349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:45.131962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:45.132060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:45.132379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:45.132656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:45.136639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:45.171722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:45.173545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e1e3d-8267-4b13-ad70-6e6c0e0af2d2 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:45.199320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 878/1759] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:45.214553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:45.215019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:45.215298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:45.252789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:45.254859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:45.299262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 882/1760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:45.332926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] MFA Rules not processed for user `dcd34aa8318b41c8be7adbbcc84c61fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:45.334156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:45.571180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:45.574268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:45.643130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85254a-a737-48ae-bb34-97d7834f5617 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:45.690360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:45.697184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:45.699227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:45.700513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] PATH_INFO: `/v3/users/bd936e8a8ee345c981c2f3a0b798ce3a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:45.700688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 879/1761] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:45.730395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:45.734319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd936e8a8ee345c981c2f3a0b798ce3a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:45.738142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:45.757936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:46.039016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd936e8a8ee345c981c2f3a0b798ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:46.040789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd936e8a8ee345c981c2f3a0b798ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:46.079022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd936e8a8ee345c981c2f3a0b798ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:46.080096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd936e8a8ee345c981c2f3a0b798ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:46.105137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65e3cd-ba6a-49b5-8334-9d780cd5c807 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:46.116830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3653a126-50ea-4bf0-9a05-a9da65624d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd936e8a8ee345c981c2f3a0b798ce3a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:46.119145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:46.121975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:46.122327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:46.125454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:46.133896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.154806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 883/1762] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:45 2022] DELETE /identity/v3/users/bd936e8a8ee345c981c2f3a0b798ce3a => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:46.156413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:46.156707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:46.156981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:46.185585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:46.187858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67901a38-7864-43aa-9049-9268bc421505 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:46.240411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 880/1763] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:46.242862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] MFA Rules not processed for user `4b1614d524d24ce486741b599268995a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:46.244646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.250450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:46.257202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:46.259494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:46.273561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:46.360696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:46.362419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.428423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 884/1764] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:46.443226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:46.449583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.449767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:46.573326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 881/1765] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:46.573784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:46.575886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.674278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaf19-0270-410b-850f-1146f06b09e8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:46.722203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:46.722894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:46.723243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:46.723534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:46.733769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042a1e7-42fe-44d8-8056-87af770c1b65 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:46.748377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:46.749782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:46.750381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:46.750975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:46.753012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:46.754866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:46.756716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.784859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 885/1766] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:00:46.801081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:46.803618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:46.930427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:46.937024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:46.970262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 882/1767] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:47.096133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc4d8f-d104-4658-a069-023d98487bc7 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:47.107531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:47.112224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:47.112963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:47.113431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:47.150525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:47.152856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:47.171326 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 886/1768] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:48.194858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:48.197515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:48.249694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532339f-2729-4d15-8f4c-50a9a48e89de service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:48.280424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:48.282217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:48.287533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:48.290845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] PATH_INFO: `/v3/projects/3ef8281ebf864fbbb863e99fa091ce0e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:48.293458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef8281ebf864fbbb863e99fa091ce0e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:48.305810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:48.451793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef8281ebf864fbbb863e99fa091ce0e', 'request_id': 'req-7e047b17-4727-4eaf-9725-6a59e699488f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:48.460193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef8281ebf864fbbb863e99fa091ce0e', 'request_id': 'req-7e047b17-4727-4eaf-9725-6a59e699488f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:48.477157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef8281ebf864fbbb863e99fa091ce0e', 'request_id': 'req-7e047b17-4727-4eaf-9725-6a59e699488f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:48.550092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] The token cache is being invalidate because project 3ef8281ebf864fbbb863e99fa091ce0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:48.551412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:48.552795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:48.557712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 883/1769] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:48 2022] DELETE /identity/v3/projects/3ef8281ebf864fbbb863e99fa091ce0e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:48.560642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:48.580379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a333d47-8bdc-48c4-a7b7-34fc0793e706 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:48.763022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:48.864670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:48.871046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:48.873575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:48.874016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] PATH_INFO: `/v3/users/17cfd847b97f4868922bc6266ccbd8ff` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:48.879958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17cfd847b97f4868922bc6266ccbd8ff)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:48.887030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:48.966887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:48.968565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:49.016126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:49.031646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e047b17-4727-4eaf-9725-6a59e699488f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:49.059473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:49.064001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:49.066297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:49.072875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] PATH_INFO: `/v3/projects/429be2da119040bca09734b9ece09b34` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:49.077290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429be2da119040bca09734b9ece09b34)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:49.085237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:49.177938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17cfd847b97f4868922bc6266ccbd8ff'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.180405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17cfd847b97f4868922bc6266ccbd8ff'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.215244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17cfd847b97f4868922bc6266ccbd8ff'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.217944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17cfd847b97f4868922bc6266ccbd8ff'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.258327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17cfd847b97f4868922bc6266ccbd8ff'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.299071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429be2da119040bca09734b9ece09b34', 'request_id': 'req-48dd0de7-f581-4c68-b0dd-809887aef2e0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.299598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 887/1770] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:48 2022] DELETE /identity/v3/users/17cfd847b97f4868922bc6266ccbd8ff => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:49.336844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429be2da119040bca09734b9ece09b34', 'request_id': 'req-48dd0de7-f581-4c68-b0dd-809887aef2e0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.356482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429be2da119040bca09734b9ece09b34', 'request_id': 'req-48dd0de7-f581-4c68-b0dd-809887aef2e0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.411402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] The token cache is being invalidate because project 429be2da119040bca09734b9ece09b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:49.411698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.412413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.424851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 884/1771] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:48 2022] DELETE /identity/v3/projects/429be2da119040bca09734b9ece09b34 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:49.426202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:49.427849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:49.634468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296cc32-049e-4be7-b3f2-482c31787e52 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:49.660284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:49.663156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:49.665103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:49.665595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] PATH_INFO: `/v3/users/c59c359a96c64e9ca3b55c6467a0c611` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:49.669345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59c359a96c64e9ca3b55c6467a0c611)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:49.672163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:49.873313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59c359a96c64e9ca3b55c6467a0c611'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.874670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59c359a96c64e9ca3b55c6467a0c611'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.900074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59c359a96c64e9ca3b55c6467a0c611'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.900916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59c359a96c64e9ca3b55c6467a0c611'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.918049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:49.930179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-2cb3a06e-87e0-4793-857b-60a619732c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59c359a96c64e9ca3b55c6467a0c611'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:49.955457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 888/1772] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:00:49 2022] DELETE /identity/v3/users/c59c359a96c64e9ca3b55c6467a0c611 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:51.523697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:51.526388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:51.763273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd0de7-f581-4c68-b0dd-809887aef2e0 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:52.089332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:52.090233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.097145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.097527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] PATH_INFO: `/v3/projects/849aae572bae4d358d22f7012c69b3b8` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.109247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849aae572bae4d358d22f7012c69b3b8)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:52.111400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:52.306041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:52.334833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849aae572bae4d358d22f7012c69b3b8', 'request_id': 'req-140b2a54-fadb-497b-ac08-03e25c484367'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:52.355407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849aae572bae4d358d22f7012c69b3b8', 'request_id': 'req-140b2a54-fadb-497b-ac08-03e25c484367'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:52.380254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849aae572bae4d358d22f7012c69b3b8', 'request_id': 'req-140b2a54-fadb-497b-ac08-03e25c484367'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:52.426715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] The token cache is being invalidate because project 849aae572bae4d358d22f7012c69b3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:52.427190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:52.429872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-140b2a54-fadb-497b-ac08-03e25c484367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:52.439479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 885/1773] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:51 2022] DELETE /identity/v3/projects/849aae572bae4d358d22f7012c69b3b8 => generated 0 bytes in 913 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:52.441987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc842f4-258b-4c65-b53d-e9e4854e40e9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.443014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc842f4-258b-4c65-b53d-e9e4854e40e9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.443488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc842f4-258b-4c65-b53d-e9e4854e40e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.466487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e474b-7e36-47b7-8516-c5854847c396 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.467068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e474b-7e36-47b7-8516-c5854847c396 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.467515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e474b-7e36-47b7-8516-c5854847c396 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.469548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 886/1774] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:00:52.480783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ecc842f4-258b-4c65-b53d-e9e4854e40e9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:52.482602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ecc842f4-258b-4c65-b53d-e9e4854e40e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:52.604921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 889/1775] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:52.610251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.612221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.612585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.618402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.618691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.618971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.663901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:52.665428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:52.689071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 887/1776] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:52.690334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:52.707592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29a5d-22ce-45dc-a88e-6bef5a794041 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:52.714264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:52.715009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.715670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.715984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.716888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:52.717248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:52.719140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:52.727236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:52.732417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:52.741978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 888/1777] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:52.743361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:52.755684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df87b-a627-4b6c-a538-c2fa8e1f7e12 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:52.770242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90e858-a07e-4526-ae8c-82433308e188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:52.771443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90e858-a07e-4526-ae8c-82433308e188 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.771729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90e858-a07e-4526-ae8c-82433308e188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.772081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90e858-a07e-4526-ae8c-82433308e188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.773250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90e858-a07e-4526-ae8c-82433308e188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:52.779654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90e858-a07e-4526-ae8c-82433308e188 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:52.833981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 889/1778] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:52 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:52.835142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:52.835429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:52.835706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:52.856988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] MFA Rules not processed for user `4b1614d524d24ce486741b599268995a`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:52.858220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:52.897544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 890/1779] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:52.898625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:52.917823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939d46b-e672-44a3-8cb5-812c8aefa544 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:52.969964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 890/1780] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:52.970950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:52.977199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:53.042756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.043696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.049162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.049162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.050224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.052551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.095599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:53.110888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 891/1781] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:52 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:53.113861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.142702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356d5a7-935b-408f-b94a-3258513db19f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.150724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.152871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.153207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.153494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.155399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.158879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.178899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 892/1782] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:53.180150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.184506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:53.186526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86487a3f-a60f-4ab9-b8f2-f9b7e90a3f67 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.202952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.203623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.204008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.204207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.211069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0315966-386b-4322-a811-1d2a40619b4e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.214122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.215785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.226369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.228335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.228814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.229668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] PATH_INFO: `/v3/projects/970f9b23187a41d7ae3054d6a3abe6df` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.231801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=970f9b23187a41d7ae3054d6a3abe6df)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.235893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.250862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 891/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:53.252494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.268387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a887-82fe-43d0-8a68-f992e34fe36a service nova] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.283221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.284389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.284688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.284966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.287111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.289189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.324303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 892/1784] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:53.329354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.364172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:53.386609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba35a2-e635-4c62-8e8c-e7375d47e7ae admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.401155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '970f9b23187a41d7ae3054d6a3abe6df', 'request_id': 'req-b4f7615e-d86a-442e-b828-13212c07adfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:53.423152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '970f9b23187a41d7ae3054d6a3abe6df', 'request_id': 'req-b4f7615e-d86a-442e-b828-13212c07adfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:53.442554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '970f9b23187a41d7ae3054d6a3abe6df', 'request_id': 'req-b4f7615e-d86a-442e-b828-13212c07adfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:53.470364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] The token cache is being invalidate because project 970f9b23187a41d7ae3054d6a3abe6df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:00:53.470658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:53.473339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b4f7615e-d86a-442e-b828-13212c07adfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:00:53.476485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 893/1785] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:00:53 2022] DELETE /identity/v3/projects/970f9b23187a41d7ae3054d6a3abe6df => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:53.483753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea540c86-197d-4a99-a9c4-1e56d71d7705 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.484048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea540c86-197d-4a99-a9c4-1e56d71d7705 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.484235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea540c86-197d-4a99-a9c4-1e56d71d7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.520531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ea540c86-197d-4a99-a9c4-1e56d71d7705 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:53.521771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ea540c86-197d-4a99-a9c4-1e56d71d7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:53.544393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ccd62-23fa-413f-bab9-27783f760603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.546738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ccd62-23fa-413f-bab9-27783f760603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.547484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ccd62-23fa-413f-bab9-27783f760603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.548122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ccd62-23fa-413f-bab9-27783f760603 admin admin] PATH_INFO: `/v3/projects/faa84a0af1944f44a98c8368bfe0e544/users/acdc77ebca544a53abf2e500316e6d8e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.587071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ccd62-23fa-413f-bab9-27783f760603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa84a0af1944f44a98c8368bfe0e544, user_id=acdc77ebca544a53abf2e500316e6d8e, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.591788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ccd62-23fa-413f-bab9-27783f760603 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.626312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 893/1786] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:53 2022] PUT /identity/v3/projects/faa84a0af1944f44a98c8368bfe0e544/users/acdc77ebca544a53abf2e500316e6d8e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:53.637656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.638009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.638213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.659750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 894/1787] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:53.667492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.667492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.667751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.688945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:53.696542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:53.700998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] MFA Rules not processed for user `acdc77ebca544a53abf2e500316e6d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:53.704864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:53.865158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 894/1788] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:53.866286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.868932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:53.887751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 895/1789] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:53.889814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.899205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f1599-f9b0-42e1-8084-835df681bbfb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.906970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.908355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.908634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.908979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.909757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.911817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.912330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e467-2e0b-498f-b97e-2ba968f04f7c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:53.920632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:53.921322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:53.921604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:53.921882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:53.922883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:00:53.923808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:53.927514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:53.955142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 895/1790] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:53 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:53.956163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:53.968859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:53.981869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 896/1791] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:00:53.983227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.000514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937b373-d3ec-4756-855e-a2027ac91189 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.043459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:54.060690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905965fa-3073-4770-8bdc-353b1fc0b70e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.068182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.068870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.069151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.069658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.070173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.073196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.104379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 896/1792] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:53 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:54.105794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.114991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.115565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.115849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.116201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.116985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.119331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.119955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c850d75-b5f2-4404-a85c-e2edcdb45d23 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.126988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.127670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.128062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.128237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.129126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.131635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.143968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 897/1793] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:54.148612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.162341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 897/1794] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:53 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:54.162573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9e12e-9a8d-4e2c-8b90-007894307efa admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.163571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.242245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231341a-2ac1-4aab-84fa-8dbe72aef0e1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.264690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.265403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.265686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.266092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.271973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.275131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.319934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 898/1795] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:54 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:54.321638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.347665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddc163-6595-4d82-9a9d-884fcd85a890 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.363039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.363901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.364199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.364528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.365343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.368881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.386904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 899/1796] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:00:54.390488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.410043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164eb23-2a0a-474b-bc59-5ad36099b190 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.416794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.417957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.418376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.418787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.419666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.422845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.443429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 900/1797] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:00:54.445522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.450936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.452228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.454883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.455320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] PATH_INFO: `/v3/projects/a1b9ed41417e4116848ce62e136c1977/users/c3f3e1fcbdfe42db9d470a5bfff93a36/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.457561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62379d9-5e80-4854-a4f7-a7ed72e13705 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.462953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407051a-3368-454b-9bb6-de9e94f0b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.464170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407051a-3368-454b-9bb6-de9e94f0b645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.464338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407051a-3368-454b-9bb6-de9e94f0b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.464896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407051a-3368-454b-9bb6-de9e94f0b645 admin admin] PATH_INFO: `/v3/projects/b4a3ac32cad9472b9713014b34855d89/users/6b56d24d30b04dae965f88bdfbb70cb3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.484620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407051a-3368-454b-9bb6-de9e94f0b645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a3ac32cad9472b9713014b34855d89, user_id=6b56d24d30b04dae965f88bdfbb70cb3, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.488280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b9ed41417e4116848ce62e136c1977, user_id=c3f3e1fcbdfe42db9d470a5bfff93a36, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.488583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407051a-3368-454b-9bb6-de9e94f0b645 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.490953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.514987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 901/1798] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:54 2022] PUT /identity/v3/projects/b4a3ac32cad9472b9713014b34855d89/users/6b56d24d30b04dae965f88bdfbb70cb3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:54.516808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.517102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.517377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.532060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 898/1799] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:54 2022] PUT /identity/v3/projects/a1b9ed41417e4116848ce62e136c1977/users/c3f3e1fcbdfe42db9d470a5bfff93a36/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:54.536597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.574927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] MFA Rules not processed for user `6b56d24d30b04dae965f88bdfbb70cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:54.577219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:54.608230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:54.687861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a136b-9d40-4b40-a64c-8f9a803126ae admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.698041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 902/1800] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:54.699332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.704633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.705555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.705955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.706329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.707170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.708911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.713569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae756-d612-472b-ab75-ce015e0591ee None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.719611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.720485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.720775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.721050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.721906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.723840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.730933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 899/1801] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:54.739721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.755772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b8d35-3db7-41aa-ad46-928d3a308f3c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:54.762904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c57d9e-9280-419f-880e-a74a15c0c637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:54.764085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c57d9e-9280-419f-880e-a74a15c0c637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.764465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c57d9e-9280-419f-880e-a74a15c0c637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.764832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c57d9e-9280-419f-880e-a74a15c0c637 admin admin] PATH_INFO: `/v3/projects/a1b9ed41417e4116848ce62e136c1977/users/c3f3e1fcbdfe42db9d470a5bfff93a36/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.767044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c57d9e-9280-419f-880e-a74a15c0c637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b9ed41417e4116848ce62e136c1977, user_id=c3f3e1fcbdfe42db9d470a5bfff93a36, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:54.768211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 903/1802] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:54 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:54.774166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c57d9e-9280-419f-880e-a74a15c0c637 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:54.781033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:54.797121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 900/1803] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:54 2022] PUT /identity/v3/projects/a1b9ed41417e4116848ce62e136c1977/users/c3f3e1fcbdfe42db9d470a5bfff93a36/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:54.798684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:54.798971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:54.799248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:54.854317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] MFA Rules not processed for user `c3f3e1fcbdfe42db9d470a5bfff93a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:54.860696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:54.930807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:54.970934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36338e-26b9-44f0-92c2-c119614e5aef admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:55.028964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 901/1804] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:55.045182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:55.049143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:55.198675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2968d-94a0-4501-af19-a1e3df924c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:55.199477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2968d-94a0-4501-af19-a1e3df924c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.199786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2968d-94a0-4501-af19-a1e3df924c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.205114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2968d-94a0-4501-af19-a1e3df924c01 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.206806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2968d-94a0-4501-af19-a1e3df924c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:55.213527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2968d-94a0-4501-af19-a1e3df924c01 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:55.270888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592a030-e104-4fee-b749-75d535c7d7bb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:55.291814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:55.293523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 904/1805] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:54 2022] POST /identity/v3/users => generated 290 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:55.298202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.300084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.300649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.301962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.302432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.303980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.364994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:55.370457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:55.371715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] MFA Rules not processed for user `4e9ad2c4909c49a6b3df8ece333e70b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:55.375435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:55.428096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 902/1806] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:55.432202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:55.463629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:55.478764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a887ba-02cb-412a-bc21-790390d4330b service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:55.501257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:55.501963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.502233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.502506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.503243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:55.505906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:55.537295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 903/1807] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:55.540220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:55.559116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 905/1808] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:55.568529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:55.572198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549e04d-e106-425d-b9d1-734d31257abb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:55.577518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:55.578426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.578857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.579836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.583179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:55.587026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:55.592498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630624-a81e-4c23-af61-7cae28e91b34 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:55.603898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 904/1809] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:55.605002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:55.617210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835b8c-16f7-4eb4-8633-e1ffe3b27bcd admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:55.619148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:55.620508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.621703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.621997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.634369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5c77f-7180-497c-a85c-0f10db1c0984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:55.635441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5c77f-7180-497c-a85c-0f10db1c0984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.638905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5c77f-7180-497c-a85c-0f10db1c0984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.640133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5c77f-7180-497c-a85c-0f10db1c0984 admin admin] PATH_INFO: `/v3/projects/0aaf0bf8fef14a1a89ea28cee2321848/users/722d3abad67843388f193828c98aff30/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.721686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5c77f-7180-497c-a85c-0f10db1c0984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aaf0bf8fef14a1a89ea28cee2321848, user_id=722d3abad67843388f193828c98aff30, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:55.725133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5c77f-7180-497c-a85c-0f10db1c0984 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:55.755308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 905/1810] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:55 2022] PUT /identity/v3/projects/0aaf0bf8fef14a1a89ea28cee2321848/users/722d3abad67843388f193828c98aff30/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:55.759460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:55.759864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:55.760228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:55.771521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:55.774480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:55.794154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 906/1811] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:00:55.828833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] MFA Rules not processed for user `722d3abad67843388f193828c98aff30`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:55.831631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:55.949647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 906/1812] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:00:55.950566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:55.966050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2935c-93ee-400f-a5c1-d9875fc23ea1 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.092787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.093543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.093763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.094042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.094758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.101491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.147745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 907/1813] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:56.152279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.168301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36693a6-d9ca-4386-ac5b-1f7be2b8ce67 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.182555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:56.268355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.269237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.269666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.270075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.270967 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.273257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.298417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 907/1814] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:00:56 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:00:56.302497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.314362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed2959-c9f9-4b73-8e8f-1681a8b3602e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.319077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.320429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.320719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.321314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.322256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.324545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.340970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 908/1815] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:56.342337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.354983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feffd9b-58f4-426c-af4d-32de10e4bf03 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.361379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.362476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.362845 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.363298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] PATH_INFO: `/v3/projects/7ec8f894e61942ae89502a3245d190e9/users/87b97a086d5341f0a8040883f1d78311/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.381918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec8f894e61942ae89502a3245d190e9, user_id=87b97a086d5341f0a8040883f1d78311, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.384669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.402587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 908/1816] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:56 2022] PUT /identity/v3/projects/7ec8f894e61942ae89502a3245d190e9/users/87b97a086d5341f0a8040883f1d78311/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:00:56.406646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.416379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e114177-b797-45d5-ac41-94a46a6ad961 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.472815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:00:56.483501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7d2ab-0656-43cd-b7cc-31175bd2f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.484266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7d2ab-0656-43cd-b7cc-31175bd2f44e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.484467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7d2ab-0656-43cd-b7cc-31175bd2f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.484741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7d2ab-0656-43cd-b7cc-31175bd2f44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.485473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7d2ab-0656-43cd-b7cc-31175bd2f44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.487193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7d2ab-0656-43cd-b7cc-31175bd2f44e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.498486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 909/1817] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:00:56.502395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.515404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096d9f6-1209-43dc-9330-7e4461743d14 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.520630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.521724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.522067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.522471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] PATH_INFO: `/v3/projects/7ec8f894e61942ae89502a3245d190e9/users/87b97a086d5341f0a8040883f1d78311/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.524663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec8f894e61942ae89502a3245d190e9, user_id=87b97a086d5341f0a8040883f1d78311, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.527449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.540648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 909/1818] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:00:56 2022] PUT /identity/v3/projects/7ec8f894e61942ae89502a3245d190e9/users/87b97a086d5341f0a8040883f1d78311/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:00:56.551861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.552171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.552447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.579611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] MFA Rules not processed for user `87b97a086d5341f0a8040883f1d78311`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:00:56.580850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:00:56.697844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 910/1819] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:00:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:00:56.709809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.770789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ff9eb-5aac-490e-bbec-d00c1df12d1b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.774632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed09f6-281a-466a-92e1-e916d94de402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.775357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed09f6-281a-466a-92e1-e916d94de402 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.775745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed09f6-281a-466a-92e1-e916d94de402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.776054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed09f6-281a-466a-92e1-e916d94de402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.786302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed09f6-281a-466a-92e1-e916d94de402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.787782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed09f6-281a-466a-92e1-e916d94de402 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.798453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 910/1820] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:00:56.840758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:00:56.850667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e80f9-8311-423e-9f79-b18ea67b2ae1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:00:56.854892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3940e-579f-420e-a6ea-252c90acb7d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:00:56.855467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3940e-579f-420e-a6ea-252c90acb7d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:00:56.855751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3940e-579f-420e-a6ea-252c90acb7d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:00:56.856112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3940e-579f-420e-a6ea-252c90acb7d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:00:56.934970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3940e-579f-420e-a6ea-252c90acb7d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:00:56.936469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3940e-579f-420e-a6ea-252c90acb7d9 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:00:56.946354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 911/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:00:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:03.311486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab3fea-7ecc-40be-a222-e380c9eda66e None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:03.312107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab3fea-7ecc-40be-a222-e380c9eda66e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:03.312107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab3fea-7ecc-40be-a222-e380c9eda66e None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:03.314307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 911/1822] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:01:03.324357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:03.324771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:03.324771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:03.358712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:03.359966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:03.446136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 912/1823] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:01:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:03.459715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab3fea-7ecc-40be-a222-e380c9eda66e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:03.462479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4cab3fea-7ecc-40be-a222-e380c9eda66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:03.487577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab3fea-7ecc-40be-a222-e380c9eda66e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:03.493809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64266b1-ab5b-4195-a769-0bd470de7872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:03.494632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64266b1-ab5b-4195-a769-0bd470de7872 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:03.494928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64266b1-ab5b-4195-a769-0bd470de7872 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:03.495206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64266b1-ab5b-4195-a769-0bd470de7872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:03.505986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64266b1-ab5b-4195-a769-0bd470de7872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:03.507505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64266b1-ab5b-4195-a769-0bd470de7872 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:03.519547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 912/1824] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:06.500882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:06.507014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:06.618392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6671e1a-ab21-4111-9c5b-936ca94c19a4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:06.629472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:06.630462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:06.630800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:06.631156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] PATH_INFO: `/v3/users/6b56d24d30b04dae965f88bdfbb70cb3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:06.633784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b56d24d30b04dae965f88bdfbb70cb3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:06.637294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:06.717566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b56d24d30b04dae965f88bdfbb70cb3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:06.718478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b56d24d30b04dae965f88bdfbb70cb3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:06.730062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b56d24d30b04dae965f88bdfbb70cb3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:06.731705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b56d24d30b04dae965f88bdfbb70cb3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:06.747038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b56d24d30b04dae965f88bdfbb70cb3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:06.762645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 913/1825] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:06 2022] DELETE /identity/v3/users/6b56d24d30b04dae965f88bdfbb70cb3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:06.767063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:06.767843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:06.767843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:06.789340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:06.790679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:06.846599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:06.899377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 913/1826] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:06.901471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:06.913533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:07.012676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4b76-f907-496b-bcd6-1b9e8335e370 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:07.016939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:07.017988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:07.018414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:07.019040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:07.037862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:07.044466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:07.065081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 914/1827] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:07.525580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:07.527797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:07.547582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ade62-f246-455d-9081-ba4a11c79b39 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:07.554068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:07.555310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:07.555825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:07.556373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] PATH_INFO: `/v3/projects/b4a3ac32cad9472b9713014b34855d89` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:07.558402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a3ac32cad9472b9713014b34855d89)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:07.561322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:07.650412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a3ac32cad9472b9713014b34855d89', 'request_id': 'req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.657507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a3ac32cad9472b9713014b34855d89', 'request_id': 'req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.668654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a3ac32cad9472b9713014b34855d89', 'request_id': 'req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.684537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] The token cache is being invalidate because project b4a3ac32cad9472b9713014b34855d89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:07.684756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.685635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.691153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 914/1828] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:07 2022] DELETE /identity/v3/projects/b4a3ac32cad9472b9713014b34855d89 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:07.694872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:07.707075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c9ff5-c991-4a08-b266-5cbd139d9c38 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:07.736957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:07.775693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:07.777206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:07.785979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:07.786952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:07.788050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:07.788352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] PATH_INFO: `/v3/users/722d3abad67843388f193828c98aff30` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:07.789914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=722d3abad67843388f193828c98aff30)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:07.792706 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:07.858028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722d3abad67843388f193828c98aff30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.858676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d78dd5-99c8-4403-95c9-31d79a1bd9c3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:07.858798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722d3abad67843388f193828c98aff30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.864013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:07.865094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:07.865417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:07.865690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] PATH_INFO: `/v3/users/ae716322b814408c881eff4058a54ded` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:07.867043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae716322b814408c881eff4058a54ded)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:07.868188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722d3abad67843388f193828c98aff30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.868952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722d3abad67843388f193828c98aff30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.869285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:07.889137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18d7fa95-ce8c-4d07-a4e4-e3b8e76f9365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722d3abad67843388f193828c98aff30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.903393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 915/1829] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:07 2022] DELETE /identity/v3/users/722d3abad67843388f193828c98aff30 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:07.925987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae716322b814408c881eff4058a54ded'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.926872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae716322b814408c881eff4058a54ded'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.927548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae716322b814408c881eff4058a54ded'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.938060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae716322b814408c881eff4058a54ded'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.956817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae716322b814408c881eff4058a54ded'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:07.962486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:07.985166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 915/1830] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:07 2022] DELETE /identity/v3/users/ae716322b814408c881eff4058a54ded => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:07.987208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:07.987657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:07.987980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:08.012578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:08.013908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:08.133081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 916/1831] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:08.160904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:08.162361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:08.238298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf464-d085-45e7-b597-69af4d1df279 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:08.242385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:08.243190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:08.243568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:08.243970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:08.254705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:08.256356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:08.268905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 916/1832] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:08.730279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:08.731847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:08.757206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c1b77-6f0b-444d-8001-770c64db33dc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:08.764719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:08.766200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:08.766486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:08.767305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] PATH_INFO: `/v3/projects/0aaf0bf8fef14a1a89ea28cee2321848` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:08.769110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aaf0bf8fef14a1a89ea28cee2321848)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:08.771986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:08.859346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:08.876470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aaf0bf8fef14a1a89ea28cee2321848', 'request_id': 'req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:08.888698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aaf0bf8fef14a1a89ea28cee2321848', 'request_id': 'req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:08.904631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aaf0bf8fef14a1a89ea28cee2321848', 'request_id': 'req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:08.926448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:08.928442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:08.932117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] The token cache is being invalidate because project 0aaf0bf8fef14a1a89ea28cee2321848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:08.932754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:08.933567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:08.935732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 917/1833] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:08 2022] DELETE /identity/v3/projects/0aaf0bf8fef14a1a89ea28cee2321848 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:08.941548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:08.946080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:08.954980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d7139-c7d2-4c20-9c0a-53d5b02b40f5 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:09.096510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:09.097759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:09.098280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:09.098772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] PATH_INFO: `/v3/users/acdc77ebca544a53abf2e500316e6d8e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:09.101166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acdc77ebca544a53abf2e500316e6d8e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:09.104295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:09.189369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90fb6d-54bb-4839-9f11-dc975a00d2a5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:09.202279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:09.202529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acdc77ebca544a53abf2e500316e6d8e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.203191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:09.203491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:09.204082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] PATH_INFO: `/v3/users/87b97a086d5341f0a8040883f1d78311` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:09.204838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acdc77ebca544a53abf2e500316e6d8e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.205554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acdc77ebca544a53abf2e500316e6d8e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.205908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b97a086d5341f0a8040883f1d78311)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:09.207925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:09.217476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acdc77ebca544a53abf2e500316e6d8e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.244283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acdc77ebca544a53abf2e500316e6d8e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.283200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 917/1834] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:08 2022] DELETE /identity/v3/users/acdc77ebca544a53abf2e500316e6d8e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:09.284348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:09.286014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:09.321474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:09.342382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b97a086d5341f0a8040883f1d78311'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.343559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b97a086d5341f0a8040883f1d78311'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.383666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b97a086d5341f0a8040883f1d78311'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.384362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b97a086d5341f0a8040883f1d78311'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.414286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-0a2c7c49-9374-4c95-b17f-d37ca80e478c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b97a086d5341f0a8040883f1d78311'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.451238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27833cc6-5604-427b-9806-842c9bedf6e3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:09.453917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 918/1835] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:08 2022] DELETE /identity/v3/users/87b97a086d5341f0a8040883f1d78311 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:09.455272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:09.455557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:09.455831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:09.467702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:09.468955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:09.470144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:09.470522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] PATH_INFO: `/v3/projects/037e84f88f8848cca8e1befb934f5744` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:09.472086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=037e84f88f8848cca8e1befb934f5744)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:09.476268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:09.502861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:09.504119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:09.571357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037e84f88f8848cca8e1befb934f5744', 'request_id': 'req-3b59a0cb-9d07-499b-b32a-997269b4ee0f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.580517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037e84f88f8848cca8e1befb934f5744', 'request_id': 'req-3b59a0cb-9d07-499b-b32a-997269b4ee0f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.594985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037e84f88f8848cca8e1befb934f5744', 'request_id': 'req-3b59a0cb-9d07-499b-b32a-997269b4ee0f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.611782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:09.644596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] The token cache is being invalidate because project 037e84f88f8848cca8e1befb934f5744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:09.644903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.647136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.649448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 918/1836] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:09 2022] DELETE /identity/v3/projects/037e84f88f8848cca8e1befb934f5744 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:09.653488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:09.656016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 919/1837] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:09.660116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:09.703754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:09.705284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:09.763743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a0cb-9d07-499b-b32a-997269b4ee0f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:09.769911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:09.770950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:09.771329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:09.771703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] PATH_INFO: `/v3/users/dcd34aa8318b41c8be7adbbcc84c61fa` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:09.774685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd34aa8318b41c8be7adbbcc84c61fa)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:09.777354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:09.797940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61e74c-2202-4b26-a708-e079f9335187 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:09.866895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:09.867911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:09.868273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:09.868562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:09.874347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:09.896316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd34aa8318b41c8be7adbbcc84c61fa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.897021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd34aa8318b41c8be7adbbcc84c61fa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.898766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd34aa8318b41c8be7adbbcc84c61fa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.901927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:09.928182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd34aa8318b41c8be7adbbcc84c61fa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.930517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:09.934646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:09.955136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd34aa8318b41c8be7adbbcc84c61fa'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:09.970568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 919/1838] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:09 2022] DELETE /identity/v3/users/dcd34aa8318b41c8be7adbbcc84c61fa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:09.974124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 920/1839] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:10.700585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:10.702068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:10.803002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d416c-e49c-4056-9e66-0c4db1589004 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:10.815519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:10.816607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:10.817425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:10.818308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] PATH_INFO: `/v3/projects/7ec8f894e61942ae89502a3245d190e9` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:10.819657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec8f894e61942ae89502a3245d190e9)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:10.822169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:10.838364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:10.854697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396b80f-5a07-4de2-b5cf-e6e77cc52735 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:10.863700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:10.865615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:10.866490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:10.868130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] PATH_INFO: `/v3/projects/f403c78428af4f5491ee0bcaa282f337` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:10.871025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f403c78428af4f5491ee0bcaa282f337)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:10.874843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:10.905180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec8f894e61942ae89502a3245d190e9', 'request_id': 'req-29eb41e8-f710-4e0f-8382-b5884a8be160'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:10.916400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec8f894e61942ae89502a3245d190e9', 'request_id': 'req-29eb41e8-f710-4e0f-8382-b5884a8be160'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:10.938045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec8f894e61942ae89502a3245d190e9', 'request_id': 'req-29eb41e8-f710-4e0f-8382-b5884a8be160'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:10.948796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] The token cache is being invalidate because project 7ec8f894e61942ae89502a3245d190e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:10.949107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:10.950052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:10.956237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 920/1840] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:10 2022] DELETE /identity/v3/projects/7ec8f894e61942ae89502a3245d190e9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:10.957873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:10.959322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:10.987438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f403c78428af4f5491ee0bcaa282f337', 'request_id': 'req-cb89cba7-d6b8-4c6d-9083-9ea54206df60'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:10.994380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f403c78428af4f5491ee0bcaa282f337', 'request_id': 'req-cb89cba7-d6b8-4c6d-9083-9ea54206df60'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.001804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f403c78428af4f5491ee0bcaa282f337', 'request_id': 'req-cb89cba7-d6b8-4c6d-9083-9ea54206df60'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.018177 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] The token cache is being invalidate because project f403c78428af4f5491ee0bcaa282f337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:11.018365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.019033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-cb89cba7-d6b8-4c6d-9083-9ea54206df60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.020965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 921/1841] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:10 2022] DELETE /identity/v3/projects/f403c78428af4f5491ee0bcaa282f337 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:11.022487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194a043-3225-44cc-a433-40834b5125a0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.022768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194a043-3225-44cc-a433-40834b5125a0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.023060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194a043-3225-44cc-a433-40834b5125a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.034291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:11.039016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb41e8-f710-4e0f-8382-b5884a8be160 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.046271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9194a043-3225-44cc-a433-40834b5125a0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:11.048356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9194a043-3225-44cc-a433-40834b5125a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.096791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:11.116232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.117192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.117473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.117749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] PATH_INFO: `/v3/projects/faa84a0af1944f44a98c8368bfe0e544` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.119388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa84a0af1944f44a98c8368bfe0e544)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.121731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.156692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 922/1842] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:11.158242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.158529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.158802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.198588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa84a0af1944f44a98c8368bfe0e544', 'request_id': 'req-2a97428c-3780-4ef6-b129-035f59b8f2eb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.203813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:11.205053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.205813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa84a0af1944f44a98c8368bfe0e544', 'request_id': 'req-2a97428c-3780-4ef6-b129-035f59b8f2eb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.222935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa84a0af1944f44a98c8368bfe0e544', 'request_id': 'req-2a97428c-3780-4ef6-b129-035f59b8f2eb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.235112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] The token cache is being invalidate because project faa84a0af1944f44a98c8368bfe0e544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:11.235610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.236547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-2a97428c-3780-4ef6-b129-035f59b8f2eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.240936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 921/1843] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:10 2022] DELETE /identity/v3/projects/faa84a0af1944f44a98c8368bfe0e544 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:11.240936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03e84a-4f7c-4534-9a51-4fe759b265be None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.240936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03e84a-4f7c-4534-9a51-4fe759b265be None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.240936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03e84a-4f7c-4534-9a51-4fe759b265be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.261036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9e03e84a-4f7c-4534-9a51-4fe759b265be None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:11.262391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9e03e84a-4f7c-4534-9a51-4fe759b265be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.305395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 923/1844] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:11.306770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.308400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.327301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ede5-fabe-418e-b794-c23b07c566e3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.334131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.335401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.336044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.336392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] PATH_INFO: `/v3/users/c3f3e1fcbdfe42db9d470a5bfff93a36` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.344632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f3e1fcbdfe42db9d470a5bfff93a36)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.348439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.357464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 922/1845] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:11.357464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.357464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.357464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.376792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:11.378250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.388583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 923/1846] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:11.389648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.402543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071dc97-2d6a-44e1-9ebb-510c68fb4b14 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.408622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.409474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.409958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.410380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.411390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:11.411841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.413644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.418189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f3e1fcbdfe42db9d470a5bfff93a36'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.418940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f3e1fcbdfe42db9d470a5bfff93a36'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.429302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 924/1847] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:11.430505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.439531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.450115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f3e1fcbdfe42db9d470a5bfff93a36'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.450962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f3e1fcbdfe42db9d470a5bfff93a36'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.458278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467940a-4d0d-46e1-90a9-22429f3a327b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.463794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.464758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.465164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.465461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.466333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:11.466663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.467315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f3e1fcbdfe42db9d470a5bfff93a36'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:11.468415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.480619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:11.486724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 925/1849] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:11.487743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.490630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:11.493789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 924/1849] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:11 2022] DELETE /identity/v3/users/c3f3e1fcbdfe42db9d470a5bfff93a36 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:11.501698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.513678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609224ff-d1ce-46a1-83a3-d69762e3a209 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.572922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:11.599709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24e60e-7475-4ffb-8e0d-960e5f74ec02 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.604803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.605600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.605801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.606077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.609151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.611715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.612182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.613102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.613387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.613666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.614394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.618513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.660106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 925/1850] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:11.665701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.677351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 926/1851] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:11.681773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.770292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f41eb-0928-413a-b655-2a787830901b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.791435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306229f-3b96-4992-9d8c-b526a0df720c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.799272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.800309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.800689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.801056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.801918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.805417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.839700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 927/1852] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:11.843689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.855593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efdd9e-8953-4397-a8e4-11c817958a19 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.861191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.861886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.862167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.862443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.863187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.865095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.878637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 928/1853] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:11.880721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.881681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.882099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.882288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.882577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.883343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.885637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.894755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ec593-0ee5-4748-9a3c-35a01658bf6e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.915210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.916254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.916793 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.917391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.920280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.923095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.928724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 926/1854] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:11 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:11.933396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.938744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 929/1855] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:11.943695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.948764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44c821-2761-462e-b113-93415a0aa51f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.954864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.956400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.956780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.957436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:11.958508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:11.962279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:11.968237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755848-0e31-4723-80e5-5cb8e06bad91 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:11.976308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 927/1856] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:11.981282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:11.983916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc5183-46ed-430b-9adc-943b1d2e0670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:11.984797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc5183-46ed-430b-9adc-943b1d2e0670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:11.985096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc5183-46ed-430b-9adc-943b1d2e0670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:11.985431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc5183-46ed-430b-9adc-943b1d2e0670 admin admin] PATH_INFO: `/v3/projects/aaaf128ed8a54d348132d43422bf82b3/users/df33a4e439a443ceb9ffa719b0cba78b/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.013362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68538355-dafe-4076-8f55-8127ce5a323d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.020298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.021622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.022369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.022855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.023845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.025132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc5183-46ed-430b-9adc-943b1d2e0670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaaf128ed8a54d348132d43422bf82b3, user_id=df33a4e439a443ceb9ffa719b0cba78b, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.027016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.027983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc5183-46ed-430b-9adc-943b1d2e0670 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.042352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 928/1857] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:12.046743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.055641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 930/1858] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:11 2022] PUT /identity/v3/projects/aaaf128ed8a54d348132d43422bf82b3/users/df33a4e439a443ceb9ffa719b0cba78b/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:12.060331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db32abc-5912-4114-b090-bfca42705cee admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.069016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e423cf-c1d8-431e-bcbf-5d13c21dbe8c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.069316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e423cf-c1d8-431e-bcbf-5d13c21dbe8c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.069608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e423cf-c1d8-431e-bcbf-5d13c21dbe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.110073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-15e423cf-c1d8-431e-bcbf-5d13c21dbe8c None None] MFA Rules not processed for user `df33a4e439a443ceb9ffa719b0cba78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:12.111702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-15e423cf-c1d8-431e-bcbf-5d13c21dbe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.120381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:12.137805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:12.149865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.152180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.152467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.152743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] PATH_INFO: `/v3/projects/7d7ca7cfa7ea480b97c08943302d2f2f/users/6d4fd0f2129a4ed29dd4ed92bce7501a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.177455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7ca7cfa7ea480b97c08943302d2f2f, user_id=6d4fd0f2129a4ed29dd4ed92bce7501a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.181999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.206846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 929/1859] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:12 2022] PUT /identity/v3/projects/7d7ca7cfa7ea480b97c08943302d2f2f/users/6d4fd0f2129a4ed29dd4ed92bce7501a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:12.207827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.217593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.237885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 931/1860] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:12.239358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.239648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.239949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.298133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] MFA Rules not processed for user `6d4fd0f2129a4ed29dd4ed92bce7501a`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:12.301314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.332959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0d2f1-39f6-487d-935b-2c2540d417d0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.338935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.339987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.340655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.341021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] PATH_INFO: `/v3/projects/a1b9ed41417e4116848ce62e136c1977` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.342339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b9ed41417e4116848ce62e136c1977)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.344473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.437398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b9ed41417e4116848ce62e136c1977', 'request_id': 'req-9fed6bec-c931-46ff-a899-446fc7aab576'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:12.453816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b9ed41417e4116848ce62e136c1977', 'request_id': 'req-9fed6bec-c931-46ff-a899-446fc7aab576'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:12.469177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 932/1861] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:12.470468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.473946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b9ed41417e4116848ce62e136c1977', 'request_id': 'req-9fed6bec-c931-46ff-a899-446fc7aab576'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:12.488546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aea5e-30fc-4f9f-97ec-f6658abc6c6a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.501401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] The token cache is being invalidate because project a1b9ed41417e4116848ce62e136c1977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:12.501700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:12.502491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:12.504626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 930/1862] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:12 2022] DELETE /identity/v3/projects/a1b9ed41417e4116848ce62e136c1977 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:12.506857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.508858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.599818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b4395-6329-4607-91be-580e137558c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.600637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b4395-6329-4607-91be-580e137558c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.601378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b4395-6329-4607-91be-580e137558c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.601748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b4395-6329-4607-91be-580e137558c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.603080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b4395-6329-4607-91be-580e137558c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.608871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b4395-6329-4607-91be-580e137558c9 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.615333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:12.623825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed6bec-c931-46ff-a899-446fc7aab576 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.633592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.634493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.634895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.635630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.636525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.638830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.642579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 933/1863] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:12.644325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.644622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.644911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.666969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:12.668431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.670572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 931/1864] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:12.671652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.673506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.720767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:12.743236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b60c0c-e906-44dd-9947-dbc60eadfd10 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.748097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6360e-b4d3-4599-a4c8-a706cf232191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.748769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6360e-b4d3-4599-a4c8-a706cf232191 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.749051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6360e-b4d3-4599-a4c8-a706cf232191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.749328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6360e-b4d3-4599-a4c8-a706cf232191 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.750047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6360e-b4d3-4599-a4c8-a706cf232191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.751130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 934/1865] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:12.751946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.752252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6360e-b4d3-4599-a4c8-a706cf232191 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.763142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f436a05-4946-449a-8a32-d69aac3bf7ea None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.768380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.769043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.769320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.769592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.770308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.772263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.773398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 932/1866] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:12.774728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.775021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.775296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.795046 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:12.796325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 935/1867] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/users => generated 298 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:12.796448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:12.797054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.807468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 933/1868] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:12.808639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.808856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc0336-7db3-4e9e-ab56-3675cb8c0788 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.814531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.815178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.815490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.815837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.816627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.818009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eed4c-abe2-4069-b771-fb8762e04fd8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.818243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.823196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.823849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.824152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.824420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.825142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.826711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.828884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 936/1869] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:12.828884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.837720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 934/1870] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:12.838584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.839401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129dc9b-5ba8-4728-8004-a983cc7acab6 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.845152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.845945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.846138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.846412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.847172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:12.847480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.849229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.850552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9218b6f-136e-49fd-a25e-3101c17827e7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.855384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.856462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.856733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.856994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] PATH_INFO: `/v3/projects/a82222aef3cb4e95af848a344c388097/users/161e6de5c59c4453b4e37381e0995172/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.862182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 937/1871] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:12.862992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.872612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82222aef3cb4e95af848a344c388097, user_id=161e6de5c59c4453b4e37381e0995172, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.872964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba91515-49fd-43ce-9f68-69a98f00d58f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.875547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.877762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.878647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.878929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.879233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] PATH_INFO: `/v3/projects/28d70a8723ee4ecaa4c73f402436dd0e/users/84ad0558047b48509360e8259ab67204/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.890461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 935/1872] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:12 2022] PUT /identity/v3/projects/a82222aef3cb4e95af848a344c388097/users/161e6de5c59c4453b4e37381e0995172/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:12.891247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.896349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d70a8723ee4ecaa4c73f402436dd0e, user_id=84ad0558047b48509360e8259ab67204, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.899051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.914893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 938/1873] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:12 2022] PUT /identity/v3/projects/28d70a8723ee4ecaa4c73f402436dd0e/users/84ad0558047b48509360e8259ab67204/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:12.917939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:12.956318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8bd8-a1e3-4589-ba9d-2067f6649443 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.979638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0820a53-123e-43b0-ab11-5e977130df97 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:12.984616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:12.985332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:12.985614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:12.985929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:12.986627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:12.988448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:12.990114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:13.001952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 939/1874] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:13.001952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.009841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382f62-5eb2-469a-a408-518fa2f56ce9 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.014934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.015640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.016026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.016157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.016247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.016597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.016876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.016966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.017145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.017856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.018535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.019731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.029014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 940/1875] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:13.029801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.041130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76364434-f884-4a9b-ad28-0be96037079e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.046207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.047119 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.047392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.047662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] PATH_INFO: `/v3/projects/a82222aef3cb4e95af848a344c388097/users/161e6de5c59c4453b4e37381e0995172/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.049614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82222aef3cb4e95af848a344c388097, user_id=161e6de5c59c4453b4e37381e0995172, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.053279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 936/1876] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:12 2022] POST /identity/v3/projects => generated 353 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:13.053279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.053490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.063837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 941/1877] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:13 2022] PUT /identity/v3/projects/a82222aef3cb4e95af848a344c388097/users/161e6de5c59c4453b4e37381e0995172/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:13.064737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.110337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:13.114421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16c52-ff41-4a90-bf4b-3cbe7e6fe2f1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.132713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c40d9-5110-4454-a1f1-593a7e333745 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.137571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93816f-4304-4768-b294-b27f4258f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.138256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93816f-4304-4768-b294-b27f4258f4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.138539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93816f-4304-4768-b294-b27f4258f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.138814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93816f-4304-4768-b294-b27f4258f4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.139545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93816f-4304-4768-b294-b27f4258f4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.141508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93816f-4304-4768-b294-b27f4258f4e5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.165390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 942/1878] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:13 2022] POST /identity/v3/users => generated 277 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:13.167104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817b8a-bf3c-4105-88d8-14e0cacf284a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.167504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817b8a-bf3c-4105-88d8-14e0cacf284a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.167888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817b8a-bf3c-4105-88d8-14e0cacf284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.176206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.177108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.177382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.177656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] PATH_INFO: `/v3/projects/28d70a8723ee4ecaa4c73f402436dd0e/users/84ad0558047b48509360e8259ab67204/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.179905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d70a8723ee4ecaa4c73f402436dd0e, user_id=84ad0558047b48509360e8259ab67204, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.182530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.194059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 937/1879] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:13 2022] PUT /identity/v3/projects/28d70a8723ee4ecaa4c73f402436dd0e/users/84ad0558047b48509360e8259ab67204/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:13.194415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e1817b8a-bf3c-4105-88d8-14e0cacf284a None None] MFA Rules not processed for user `161e6de5c59c4453b4e37381e0995172`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:13.195129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.196072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e1817b8a-bf3c-4105-88d8-14e0cacf284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.265642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017034-8691-4202-b7cb-31408ed802e4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.270458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.271231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.271625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.271960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.272728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.274070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 943/1880] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:13.274499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.275318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.275704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.276061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.277103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:13.288247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 938/1881] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:13.292522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.304382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333cb4-0b5b-47cb-a6f8-d7a79260153c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.309914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] MFA Rules not processed for user `84ad0558047b48509360e8259ab67204`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:13.310534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.311123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.311686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.312254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.312672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.313567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.315645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.330710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 939/1882] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:13.332555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.334919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.394889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 944/1883] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:13.395725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.407488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14d497-3b05-4f63-8eff-029b32d60607 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.409433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba254b7f-5a10-482a-9381-ed1aac2ec311 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.411438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.412239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.412552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.412890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.414191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd510d08-7d8a-4d19-b283-e73a4ad0a9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.415082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510d08-7d8a-4d19-b283-e73a4ad0a9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.415357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510d08-7d8a-4d19-b283-e73a4ad0a9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.415620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510d08-7d8a-4d19-b283-e73a4ad0a9c4 admin admin] PATH_INFO: `/v3/projects/fcfc1612a14d4b13af7da45fd6171fe4/users/51d401bbf09b4dc7923a1826a976ff17/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.423902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.425893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.436485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd510d08-7d8a-4d19-b283-e73a4ad0a9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfc1612a14d4b13af7da45fd6171fe4, user_id=51d401bbf09b4dc7923a1826a976ff17, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.439480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd510d08-7d8a-4d19-b283-e73a4ad0a9c4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.447585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 940/1884] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:13.448986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.451132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.464602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36d45-2234-400c-8c88-3436a22cb89b service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.466034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 945/1885] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:13 2022] PUT /identity/v3/projects/fcfc1612a14d4b13af7da45fd6171fe4/users/51d401bbf09b4dc7923a1826a976ff17/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:13.479407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.480770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.482013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.526057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] MFA Rules not processed for user `51d401bbf09b4dc7923a1826a976ff17`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:13.527604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.532416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:13.544731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.545535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.546114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.546489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.613925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.614528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 946/1886] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:13.618084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.618182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.624952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887efaa-f8d4-484b-9d5a-026129348904 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.626413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 941/1887] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:13.627307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.629147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.629791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.630075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.630348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.632202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.675257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:13.703278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.704877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.707413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a94931-130b-4881-9281-de05db588c5b service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.713808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.714077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.714429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.714893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.715982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.718878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.729925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 947/1888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:13.746295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.757505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b87da-0070-4c2a-a708-cac2e04c006b service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.766194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.766873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.767157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.767419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.769592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:13.781270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 942/1889] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:13 2022] POST /identity/v3/projects => generated 355 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:13.783101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.843977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.845591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.872841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 948/1890] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:13.873367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798cfb-19d9-426f-8dfe-26feeef40ae7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.879184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.880200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.880491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.880767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.882107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.884126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.910187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 943/1891] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:13 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:13.914491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.937014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d4d71-c9a0-41fe-97a9-0d337c525c38 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:13.968319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:13.969079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:13.969363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:13.969637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:13.970378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:13.972021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:13.984795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 949/1892] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:13.988932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:13.999609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef750d6-8903-4193-977a-700a0810499c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:14.005198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:14.006521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:14.006858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:14.007272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] PATH_INFO: `/v3/projects/8cbda8c6a64d4f69b53887026f0d6221/users/117df7b26d1848078a0881df7c889c51/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:14.026723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbda8c6a64d4f69b53887026f0d6221, user_id=117df7b26d1848078a0881df7c889c51, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:14.029728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:14.051353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 944/1893] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:13 2022] PUT /identity/v3/projects/8cbda8c6a64d4f69b53887026f0d6221/users/117df7b26d1848078a0881df7c889c51/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:14.055456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:14.066781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecf403-586a-44bd-9903-909326e7dd90 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:14.111538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:14.144659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbde181-b8c7-40a3-9578-0f8e0c9d462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:14.145474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbde181-b8c7-40a3-9578-0f8e0c9d462b admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:14.145849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbde181-b8c7-40a3-9578-0f8e0c9d462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:14.146162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbde181-b8c7-40a3-9578-0f8e0c9d462b admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:14.146973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbde181-b8c7-40a3-9578-0f8e0c9d462b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:14.148627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbde181-b8c7-40a3-9578-0f8e0c9d462b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:14.168493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 950/1894] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:14.173016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:14.184748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ae83-25ea-4854-ade8-24b54dd6b017 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:14.191169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:14.192501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:14.192837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:14.193122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] PATH_INFO: `/v3/projects/8cbda8c6a64d4f69b53887026f0d6221/users/117df7b26d1848078a0881df7c889c51/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:14.195482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbda8c6a64d4f69b53887026f0d6221, user_id=117df7b26d1848078a0881df7c889c51, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:14.198971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:14.210440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 945/1895] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:14 2022] PUT /identity/v3/projects/8cbda8c6a64d4f69b53887026f0d6221/users/117df7b26d1848078a0881df7c889c51/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:14.220726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:14.221108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:14.221281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:14.250237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] MFA Rules not processed for user `117df7b26d1848078a0881df7c889c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:14.251472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:14.337557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 951/1896] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:14 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:14.349311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:14.406247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622bdf4-e083-4c57-8652-a58a08e0b4f4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:14.410512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd00ba6-0176-48ff-9e19-8e7319c23538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:14.411127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd00ba6-0176-48ff-9e19-8e7319c23538 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:14.411401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd00ba6-0176-48ff-9e19-8e7319c23538 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:14.411671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd00ba6-0176-48ff-9e19-8e7319c23538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:14.421818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd00ba6-0176-48ff-9e19-8e7319c23538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:14.423289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd00ba6-0176-48ff-9e19-8e7319c23538 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:14.433247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 946/1897] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:14.512994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:14.524021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f10f7d-8f9b-4e98-b3c7-f6aade31cff9 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:14.528484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143ede3-1a98-472b-8333-54b198d6280e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:14.529129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ede3-1a98-472b-8333-54b198d6280e service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:14.529407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ede3-1a98-472b-8333-54b198d6280e service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:14.529677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ede3-1a98-472b-8333-54b198d6280e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:14.602323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143ede3-1a98-472b-8333-54b198d6280e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:14.603890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143ede3-1a98-472b-8333-54b198d6280e service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:14.614682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 952/1898] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:17.665163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47b34-e4e1-4994-a727-28544ed40451 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:17.666026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47b34-e4e1-4994-a727-28544ed40451 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:17.669033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47b34-e4e1-4994-a727-28544ed40451 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:17.670968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 947/1899] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:01:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:01:17.691674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:17.692583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:17.692792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:17.714676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:17.716564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:17.774390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:17.804802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 953/1900] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:17.806035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f47b34-e4e1-4994-a727-28544ed40451 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:17.839676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:17.841481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:17.856219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:17.869471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3fed9-7b51-48de-8e97-c9ba1b33076d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:17.878439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57db1b-251a-4178-89dc-04ba86e1fd57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:17.879681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57db1b-251a-4178-89dc-04ba86e1fd57 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:17.880150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57db1b-251a-4178-89dc-04ba86e1fd57 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:17.880645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57db1b-251a-4178-89dc-04ba86e1fd57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:17.896059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57db1b-251a-4178-89dc-04ba86e1fd57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:17.898090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57db1b-251a-4178-89dc-04ba86e1fd57 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:17.922243 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 954/1901] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:17.963906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f47b34-e4e1-4994-a727-28544ed40451 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:17.972305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:17.974320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:17.974804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:17.975249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] PATH_INFO: `/v3/users/6d4fd0f2129a4ed29dd4ed92bce7501a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:17.977320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4fd0f2129a4ed29dd4ed92bce7501a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:17.984682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:18.085496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4fd0f2129a4ed29dd4ed92bce7501a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:18.086510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4fd0f2129a4ed29dd4ed92bce7501a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:18.105308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4fd0f2129a4ed29dd4ed92bce7501a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:18.106679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4fd0f2129a4ed29dd4ed92bce7501a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:18.155863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4fd0f2129a4ed29dd4ed92bce7501a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:18.194029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 948/1902] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:17 2022] DELETE /identity/v3/users/6d4fd0f2129a4ed29dd4ed92bce7501a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:18.199550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:18.199944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:18.200304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:18.224826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:18.226088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:18.466109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 955/1903] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:18.478525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:18.480774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:18.609042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f19446-648a-4be0-9697-d48bb8122fba admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:18.615698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:18.616790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:18.617903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:18.618602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:18.643232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:18.648215 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:18.679641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 949/1904] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:19.221481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:19.223091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:19.251809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f3ee7-29aa-41ac-9f39-53c6bbc88603 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:19.277785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:19.278691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:19.278971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:19.279300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] PATH_INFO: `/v3/projects/7d7ca7cfa7ea480b97c08943302d2f2f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:19.280554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7ca7cfa7ea480b97c08943302d2f2f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:19.282485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:19.365788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7ca7cfa7ea480b97c08943302d2f2f', 'request_id': 'req-c7027824-6a6a-43dc-8685-141031bb1418'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.371702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7ca7cfa7ea480b97c08943302d2f2f', 'request_id': 'req-c7027824-6a6a-43dc-8685-141031bb1418'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.380265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7ca7cfa7ea480b97c08943302d2f2f', 'request_id': 'req-c7027824-6a6a-43dc-8685-141031bb1418'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.392616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] The token cache is being invalidate because project 7d7ca7cfa7ea480b97c08943302d2f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:19.393121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.393930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.395756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 956/1905] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:19 2022] DELETE /identity/v3/projects/7d7ca7cfa7ea480b97c08943302d2f2f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:19.399836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:19.410321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831807a0-6496-4964-a749-6b7d99543112 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:19.455425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:19.474771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:19.475904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:19.476187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:19.476667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] PATH_INFO: `/v3/users/84ad0558047b48509360e8259ab67204` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:19.477916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ad0558047b48509360e8259ab67204)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:19.480133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:19.547813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ad0558047b48509360e8259ab67204'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.548769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ad0558047b48509360e8259ab67204'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.558126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ad0558047b48509360e8259ab67204'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.558750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ad0558047b48509360e8259ab67204'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.578127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ad0558047b48509360e8259ab67204'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:19.591530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 950/1906] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:19 2022] DELETE /identity/v3/users/84ad0558047b48509360e8259ab67204 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:19.878375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:19.879532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:19.893125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:19.894638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:19.910562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7027824-6a6a-43dc-8685-141031bb1418 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:19.973334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cca794-bb29-4787-8c1f-2808970d5829 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:19.978957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:19.980037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:19.980614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:19.981089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] PATH_INFO: `/v3/projects/28d70a8723ee4ecaa4c73f402436dd0e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:19.982530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d70a8723ee4ecaa4c73f402436dd0e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:19.983951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:19.984750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:19.998896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.000480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.000480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.000670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] PATH_INFO: `/v3/users/fb4900ec7bcf4ec9ae115e8e81c60de1` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.002168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4900ec7bcf4ec9ae115e8e81c60de1)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.004697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.048684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d70a8723ee4ecaa4c73f402436dd0e', 'request_id': 'req-ac432dd9-19ae-46c6-9538-80049b580111'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.055305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d70a8723ee4ecaa4c73f402436dd0e', 'request_id': 'req-ac432dd9-19ae-46c6-9538-80049b580111'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.062402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d70a8723ee4ecaa4c73f402436dd0e', 'request_id': 'req-ac432dd9-19ae-46c6-9538-80049b580111'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.064693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:20.073343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4900ec7bcf4ec9ae115e8e81c60de1'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.074071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4900ec7bcf4ec9ae115e8e81c60de1'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.074889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4900ec7bcf4ec9ae115e8e81c60de1'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.080453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] The token cache is being invalidate because project 28d70a8723ee4ecaa4c73f402436dd0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:20.081035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.082220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ac432dd9-19ae-46c6-9538-80049b580111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.084378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 951/1907] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:19 2022] DELETE /identity/v3/projects/28d70a8723ee4ecaa4c73f402436dd0e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:20.093897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fedfc-3281-4096-8ac3-dcee1ade800d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.093897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fedfc-3281-4096-8ac3-dcee1ade800d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.093897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fedfc-3281-4096-8ac3-dcee1ade800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.095433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4900ec7bcf4ec9ae115e8e81c60de1'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.112904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-0f6fedfc-3281-4096-8ac3-dcee1ade800d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:20.114006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e29ad6c1-8b58-4a0b-bbab-c9b7aa389e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4900ec7bcf4ec9ae115e8e81c60de1'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.114363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0f6fedfc-3281-4096-8ac3-dcee1ade800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.138201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 957/1908] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:19 2022] DELETE /identity/v3/users/fb4900ec7bcf4ec9ae115e8e81c60de1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:20.143010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.143937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.144367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.165186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:20.167495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.189731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 952/1909] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:20.201507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.201792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.202067 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.232196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:20.233416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.236893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 958/1910] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:20.246825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.248627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.252022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 953/1911] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:20.253824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.268686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03d26b-e5e3-43ab-a3ca-8b6609afb132 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.278830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.283556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.283981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.284354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.285241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:20.285555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.288799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.308591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 954/1912] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:20.314885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.336889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7709-6d40-4017-8019-025082f6b7ba admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.345109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.346137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.346599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.347064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.348030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.348615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a563bd-92f5-4e18-80e6-2f8820bafd6e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.351737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.358959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.359692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.361780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.362016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.404729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.406490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.426135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 959/1913] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:20.442460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 955/1914] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:20.447788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.462512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d51173-9597-44cc-9308-03410170bde4 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.478538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:20.551932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.553699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.554323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.554840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.555903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.558897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.587274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 960/1915] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:20.591798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.608498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc294a39-54d9-453b-ad47-47661e56d7ae admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.616895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.620877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.620877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.620877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.620877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.621599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.625976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.630643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.638905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f527f-c0a3-4064-ac0e-6d35466d7914 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.645343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.646365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.646675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.646957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] PATH_INFO: `/v3/users/df33a4e439a443ceb9ffa719b0cba78b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.648477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=df33a4e439a443ceb9ffa719b0cba78b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.651195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 956/1916] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:20.654220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.657107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.670571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622599a-830a-4019-97f9-a48642fbaef7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.677079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.678267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.678755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.679207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.680306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.689540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.714734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 957/1917] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:20.716075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:20.734352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65057-5441-4c4a-8db3-0a2ddab6ef2f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:20.740064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872fb1a-7416-4cdf-9404-d34b9438b1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:20.741025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872fb1a-7416-4cdf-9404-d34b9438b1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.741371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872fb1a-7416-4cdf-9404-d34b9438b1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.741584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872fb1a-7416-4cdf-9404-d34b9438b1b3 admin admin] PATH_INFO: `/v3/projects/7a9f17d6ec6a4ecb93286597f8ae56e5/users/2a79f9e93a984005ba6a41deb62877e0/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.761465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872fb1a-7416-4cdf-9404-d34b9438b1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9f17d6ec6a4ecb93286597f8ae56e5, user_id=2a79f9e93a984005ba6a41deb62877e0, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:20.767941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872fb1a-7416-4cdf-9404-d34b9438b1b3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:20.778050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df33a4e439a443ceb9ffa719b0cba78b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.778987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df33a4e439a443ceb9ffa719b0cba78b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.780296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df33a4e439a443ceb9ffa719b0cba78b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.788435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 958/1918] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:20 2022] PUT /identity/v3/projects/7a9f17d6ec6a4ecb93286597f8ae56e5/users/2a79f9e93a984005ba6a41deb62877e0/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:20.798206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df33a4e439a443ceb9ffa719b0cba78b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.802662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.802952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.803232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.816963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-116fba5a-18b8-4624-b068-983d9ee0cecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df33a4e439a443ceb9ffa719b0cba78b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:20.832562 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 961/1919] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:20 2022] DELETE /identity/v3/users/df33a4e439a443ceb9ffa719b0cba78b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:20.837204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:20.837495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:20.839454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:20.860877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] MFA Rules not processed for user `2a79f9e93a984005ba6a41deb62877e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:20.868434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.879064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:20.879817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:20.992976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:21.006302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 962/1920] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:21.019190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.019190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.019806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 959/1921] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:21.020892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.022540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.038970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6556bd-9536-4ca1-965f-8013c88de79a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.045644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.046661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.047121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.047606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.048883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.049762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:21.050822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.092081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 960/1922] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:21 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:21.093380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.095766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.103328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55215017-bc2b-4e51-94bf-817fe661a098 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.179147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4505a8d-3ad5-4907-919d-36e1a4584f69 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.182599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.183267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.183547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.183821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.184526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.185404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.185680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.186094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] PATH_INFO: `/v3/projects/3e58d76101894cafbe0f69e7c8d6891e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.188005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e58d76101894cafbe0f69e7c8d6891e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.190165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.197541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.198786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.213162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 963/1923] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:21.231860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.245906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99400157-740c-4367-bb6c-fb44688d1cdb service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.252051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.253061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.253223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.253825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.254870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.257590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.275329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e58d76101894cafbe0f69e7c8d6891e', 'request_id': 'req-eb6c80b5-8a74-47e6-8da1-62f2a1880159'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.287813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e58d76101894cafbe0f69e7c8d6891e', 'request_id': 'req-eb6c80b5-8a74-47e6-8da1-62f2a1880159'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.307797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e58d76101894cafbe0f69e7c8d6891e', 'request_id': 'req-eb6c80b5-8a74-47e6-8da1-62f2a1880159'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.309660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 964/1924] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:21 2022] POST /identity/v3/users => generated 300 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:21.313917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.333640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] The token cache is being invalidate because project 3e58d76101894cafbe0f69e7c8d6891e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:21.333938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.334736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.334888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e55c10-6b58-4113-8bd5-77fa6f8d0948 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.336639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 961/1925] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:21 2022] DELETE /identity/v3/projects/3e58d76101894cafbe0f69e7c8d6891e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:21.340606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.343108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.421770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c80b5-8a74-47e6-8da1-62f2a1880159 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.426524 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.426877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.427297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.427627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.427774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.428068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.428159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.428343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] PATH_INFO: `/v3/users/19777803437747fabb676dc950009e14` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.428812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.429583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19777803437747fabb676dc950009e14)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.430426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.431575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.435521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:21.442992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 965/1926] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:21.448659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.450622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.465450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa23dc-9d4d-4b34-bdfb-72dfe8cffeea admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.470710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.471794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.472172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.472928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] PATH_INFO: `/v3/projects/009e89806cce42409aad807d5d8838b2/users/743fffd50e714a16b9d7dfe4609999d0/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.490341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009e89806cce42409aad807d5d8838b2, user_id=743fffd50e714a16b9d7dfe4609999d0, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.493067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.493288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19777803437747fabb676dc950009e14'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.493948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19777803437747fabb676dc950009e14'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.504325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19777803437747fabb676dc950009e14'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.505105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19777803437747fabb676dc950009e14'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.510043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 966/1927] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:21 2022] PUT /identity/v3/projects/009e89806cce42409aad807d5d8838b2/users/743fffd50e714a16b9d7dfe4609999d0/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:21.514189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.516255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.520025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19777803437747fabb676dc950009e14'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.538486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 962/1928] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:21 2022] DELETE /identity/v3/users/19777803437747fabb676dc950009e14 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:21.560425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:21.561973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.564374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.600329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34b968-6390-4529-897b-afcc6297ba68 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.671788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d3d4-9696-452a-bd4a-0b2bd0d6e693 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.679631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.684160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.684725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.685174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] PATH_INFO: `/v3/projects/aaaf128ed8a54d348132d43422bf82b3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.685663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.686443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.686760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.687116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.688449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.688894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaaf128ed8a54d348132d43422bf82b3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.690265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.692623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.707549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 967/1929] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:21.709011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.728000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2c69-3c14-46e9-ac65-1d694074bce2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.741476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceee42-7e8b-4583-bc3e-c49d808780ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.742659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceee42-7e8b-4583-bc3e-c49d808780ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.743107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceee42-7e8b-4583-bc3e-c49d808780ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.743392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceee42-7e8b-4583-bc3e-c49d808780ce admin admin] PATH_INFO: `/v3/projects/009e89806cce42409aad807d5d8838b2/users/743fffd50e714a16b9d7dfe4609999d0/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.745644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceee42-7e8b-4583-bc3e-c49d808780ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=009e89806cce42409aad807d5d8838b2, user_id=743fffd50e714a16b9d7dfe4609999d0, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.748405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceee42-7e8b-4583-bc3e-c49d808780ce admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:21.778199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 968/1930] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:21 2022] PUT /identity/v3/projects/009e89806cce42409aad807d5d8838b2/users/743fffd50e714a16b9d7dfe4609999d0/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:21.782901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaaf128ed8a54d348132d43422bf82b3', 'request_id': 'req-98eea267-124a-4251-9cee-2b4c02d6edc8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.791210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.791566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.791839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.795386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaaf128ed8a54d348132d43422bf82b3', 'request_id': 'req-98eea267-124a-4251-9cee-2b4c02d6edc8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.806769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaaf128ed8a54d348132d43422bf82b3', 'request_id': 'req-98eea267-124a-4251-9cee-2b4c02d6edc8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.828762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] The token cache is being invalidate because project aaaf128ed8a54d348132d43422bf82b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:21.829932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.830053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] MFA Rules not processed for user `743fffd50e714a16b9d7dfe4609999d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:21.831283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:21.831569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.833675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 963/1931] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:21 2022] DELETE /identity/v3/projects/aaaf128ed8a54d348132d43422bf82b3 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:21.837489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.842722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.876527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:21.939926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 969/1932] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:21.948986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:21.950854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:21.960692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eea267-124a-4251-9cee-2b4c02d6edc8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:21.966854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:21.967763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:21.968075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:21.968352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] PATH_INFO: `/v3/users/161e6de5c59c4453b4e37381e0995172` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:21.972098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=161e6de5c59c4453b4e37381e0995172)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:21.974437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:22.046829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '161e6de5c59c4453b4e37381e0995172'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.047833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '161e6de5c59c4453b4e37381e0995172'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.052337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:22.060243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '161e6de5c59c4453b4e37381e0995172'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.062458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '161e6de5c59c4453b4e37381e0995172'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.065058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee60a07-4b45-427f-a73f-f6773d1f2f71 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:22.077667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:22.078329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.078599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.079072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.080244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '161e6de5c59c4453b4e37381e0995172'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.085910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:22.094970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 964/1933] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:21 2022] DELETE /identity/v3/users/161e6de5c59c4453b4e37381e0995172 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:22.113339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:22.115197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a45f0-1c06-48d4-a20c-669356cd3b08 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:22.129843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 970/1934] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:22.223416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:22.224921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:22.350602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b53f-7616-4425-a6e0-bd6d4513d4f2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:22.356528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:22.357544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.357831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.358107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] PATH_INFO: `/v3/projects/df81298ea61a44fc8df32f58b178fdcb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.359373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df81298ea61a44fc8df32f58b178fdcb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:22.361380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:22.453256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:22.482099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df81298ea61a44fc8df32f58b178fdcb', 'request_id': 'req-c693cf8f-73fe-4bf9-8704-033284f03776'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.489126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df81298ea61a44fc8df32f58b178fdcb', 'request_id': 'req-c693cf8f-73fe-4bf9-8704-033284f03776'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.498455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df81298ea61a44fc8df32f58b178fdcb', 'request_id': 'req-c693cf8f-73fe-4bf9-8704-033284f03776'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.525517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] The token cache is being invalidate because project df81298ea61a44fc8df32f58b178fdcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:22.525791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.526520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c693cf8f-73fe-4bf9-8704-033284f03776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:22.531912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 965/1935] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:22 2022] DELETE /identity/v3/projects/df81298ea61a44fc8df32f58b178fdcb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:22.534517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.534965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.535057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.563833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:22.564690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:22.656877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 971/1936] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:22.658615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.658913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.659197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.684254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:22.686002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:22.694769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 966/1937] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:22.699548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:22.714388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253f754-0c19-4361-8700-f5744fdcdbbb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:22.720689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:22.721848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.722334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.722441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.723363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:22.723681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:22.725385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:22.743913 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 972/1938] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:22.748349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:22.769625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdc51b-56f8-4b7e-bf18-1366cc5ac3c9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:22.793676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:22.794537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.794965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.795453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.797127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:22.799423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:22.824078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:22.826344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:22.853518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 967/1939] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:22 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:22.854689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:22.898755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6265876-62e8-4b53-b1ea-794b8e83ef38 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:22.981480 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4b52-0518-498c-bc03-07b0d5aa30ac admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:22.987671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:22.988810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.989263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.989733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.990706 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:22.992162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:22.993215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:22.993532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:22.993652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:22.994043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] PATH_INFO: `/v3/projects/a82222aef3cb4e95af848a344c388097` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:22.995551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82222aef3cb4e95af848a344c388097)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:22.997667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.020349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 968/1940] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:22 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:23.025032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.036940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc93b0-5277-4037-b3b1-0cc4456a0cd9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.046938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.047715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.047964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.048314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.049055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.050668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.062051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 969/1941] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:23.062889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.064671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.068107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82222aef3cb4e95af848a344c388097', 'request_id': 'req-e1cf7ca0-09a4-4b6b-b654-45434250ed13'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.073874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82222aef3cb4e95af848a344c388097', 'request_id': 'req-e1cf7ca0-09a4-4b6b-b654-45434250ed13'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.078459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e60a8a-a5e0-4c90-b859-fdcf9a1410c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.081939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82222aef3cb4e95af848a344c388097', 'request_id': 'req-e1cf7ca0-09a4-4b6b-b654-45434250ed13'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.094740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] The token cache is being invalidate because project a82222aef3cb4e95af848a344c388097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:23.095026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.095656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.097431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 973/1942] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:22 2022] DELETE /identity/v3/projects/a82222aef3cb4e95af848a344c388097 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:23.097972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.099226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.110277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:23.143554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.144551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.144837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.145111 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] PATH_INFO: `/v3/users/2a79f9e93a984005ba6a41deb62877e0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.146327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a79f9e93a984005ba6a41deb62877e0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.148300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.161335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:23.169187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf7ca0-09a4-4b6b-b654-45434250ed13 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.174219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79899dff-8cd3-4e9f-b7d9-708c8df11b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.175007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79899dff-8cd3-4e9f-b7d9-708c8df11b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.175369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79899dff-8cd3-4e9f-b7d9-708c8df11b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.175644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79899dff-8cd3-4e9f-b7d9-708c8df11b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.176431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79899dff-8cd3-4e9f-b7d9-708c8df11b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.178258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79899dff-8cd3-4e9f-b7d9-708c8df11b1a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.188868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 974/1943] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:23.190108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.190393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.190667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.202636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a79f9e93a984005ba6a41deb62877e0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.203228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a79f9e93a984005ba6a41deb62877e0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.209432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:23.210652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.211847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a79f9e93a984005ba6a41deb62877e0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.212603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a79f9e93a984005ba6a41deb62877e0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.219743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 975/1944] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:23.220574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.221934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.233482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9d9116a0-52a5-42fb-8fd0-55627cafc053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a79f9e93a984005ba6a41deb62877e0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:23.236787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf598a-d684-4999-b5ac-22b1940ddecf None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.243652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf94c8c-4c38-4355-ae7e-12ac0cfc2f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.244957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf94c8c-4c38-4355-ae7e-12ac0cfc2f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.245237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf94c8c-4c38-4355-ae7e-12ac0cfc2f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.245576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf94c8c-4c38-4355-ae7e-12ac0cfc2f4d admin admin] PATH_INFO: `/v3/projects/4ee0735ab2cf469c8662621a6740ca9e/users/2c088c79bbf4444a981d82dbd8a4b935/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.247359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 970/1945] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:23 2022] DELETE /identity/v3/users/2a79f9e93a984005ba6a41deb62877e0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:23.248441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0062802-3c10-40b3-81c2-79c550db8437 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.248735 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0062802-3c10-40b3-81c2-79c550db8437 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.249007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0062802-3c10-40b3-81c2-79c550db8437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.268702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d0062802-3c10-40b3-81c2-79c550db8437 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:23.269087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf94c8c-4c38-4355-ae7e-12ac0cfc2f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee0735ab2cf469c8662621a6740ca9e, user_id=2c088c79bbf4444a981d82dbd8a4b935, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.270147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d0062802-3c10-40b3-81c2-79c550db8437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.272507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf94c8c-4c38-4355-ae7e-12ac0cfc2f4d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.309112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 976/1946] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:23 2022] PUT /identity/v3/projects/4ee0735ab2cf469c8662621a6740ca9e/users/2c088c79bbf4444a981d82dbd8a4b935/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:23.310315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.310601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.310916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.329397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:23.330593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.345750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 971/1947] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:23.350469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.350890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.351235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.379541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] MFA Rules not processed for user `2c088c79bbf4444a981d82dbd8a4b935`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:23.380802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.394483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 977/1948] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:23.395489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.411676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1257b86-3aa7-42ea-a15d-9fb048aea4e1 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.416586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.417371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.417778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.418060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.418787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:23.419097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.420729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.434290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 978/1949] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:23.435140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.436968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:23.463638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 972/1950] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:23.468101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.481344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5408cc-882c-41e0-b897-b8f2e449ae98 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.486805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.488294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.488838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.489151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.490085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.492158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.506242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae0b8-be18-40d6-acce-1445035f75b5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.510010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.511021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.511322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.511799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.514163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:23.523098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.525053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 973/1951] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:23.526321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.528122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.539862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 979/1952] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:23.545052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.553355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:23.561874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f87b1a-84cc-479b-947e-567867e2fd37 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.627704 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cd82c-bd34-4f09-bcd5-75cde6e72303 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.633142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.634510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.634510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.634901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.635742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.638895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.667863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.668981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.669370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.669710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.670757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.673315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.677643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 974/1953] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:23.682212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.699692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 980/1954] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:23.703685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.717646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0325799-14f5-4456-b439-85c2b6e8952c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.826380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52222a86-0f1a-46a3-814a-3d50446e8b5d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.841810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.842780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.843208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.843628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.844761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.845731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.846508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.846756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.847028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.847762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.851303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.851721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.902829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 975/1955] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:23 2022] POST /identity/v3/users => generated 290 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:23.908436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.937494 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35528245-c24e-4561-9854-1dc0f048b710 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:23.951361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:23.952416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:23.953022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:23.953518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:23.955858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:23.957492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 981/1956] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:23.960371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:23.960505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.984432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 976/1957] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:23.987416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:23.999966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1f70-82ab-43dc-add2-c37bc959f6dc admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.002777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f019f38-adc0-42d2-a7d6-3a3c2f5193e6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.012750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.014753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.015470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.016371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] PATH_INFO: `/v3/projects/fb5540263ab94b67a5f08f21de16ed3d/users/8ca0bff7c64c437d9ce2004d264fcf33/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.020428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.021151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.021482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.021776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.022551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.037739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.071148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5540263ab94b67a5f08f21de16ed3d, user_id=8ca0bff7c64c437d9ce2004d264fcf33, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.074276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 982/1958] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:24.074438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.079013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.105365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 977/1959] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:23 2022] PUT /identity/v3/projects/fb5540263ab94b67a5f08f21de16ed3d/users/8ca0bff7c64c437d9ce2004d264fcf33/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:24.106088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cac286-6cf7-4f63-8f3d-11866e88a7c6 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.106533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.182957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:24.234024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ace045-e5b4-46b2-89c4-f912d1354ef4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.237111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68584cb-3345-4d00-bbd6-a5a135db4ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.238778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68584cb-3345-4d00-bbd6-a5a135db4ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.238778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68584cb-3345-4d00-bbd6-a5a135db4ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.238778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68584cb-3345-4d00-bbd6-a5a135db4ea9 admin admin] PATH_INFO: `/v3/projects/5fa779639da44c10aa263e2780f8e6bf/users/bf363ab070394b04b49ed3abd97d48c0/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.251765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.252779 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.253208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.253631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.254634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.257013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.261911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68584cb-3345-4d00-bbd6-a5a135db4ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa779639da44c10aa263e2780f8e6bf, user_id=bf363ab070394b04b49ed3abd97d48c0, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.264729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68584cb-3345-4d00-bbd6-a5a135db4ea9 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.280537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:24.286181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 983/1960] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:24 2022] PUT /identity/v3/projects/5fa779639da44c10aa263e2780f8e6bf/users/bf363ab070394b04b49ed3abd97d48c0/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:24.300066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 978/1961] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:24.305733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023bd7-690f-4c74-a0f6-d49ed3b59729 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.305926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.306185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023bd7-690f-4c74-a0f6-d49ed3b59729 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.306870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023bd7-690f-4c74-a0f6-d49ed3b59729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.336048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaacdb-4edb-4845-915a-6825a1a2ea27 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.356683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-05023bd7-690f-4c74-a0f6-d49ed3b59729 None None] MFA Rules not processed for user `bf363ab070394b04b49ed3abd97d48c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:24.357850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-05023bd7-690f-4c74-a0f6-d49ed3b59729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.415266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.416569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.416795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.417127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] PATH_INFO: `/v3/projects/fb5540263ab94b67a5f08f21de16ed3d/users/8ca0bff7c64c437d9ce2004d264fcf33/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.422750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5540263ab94b67a5f08f21de16ed3d, user_id=8ca0bff7c64c437d9ce2004d264fcf33, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.422750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.438992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 979/1962] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:24 2022] PUT /identity/v3/projects/fb5540263ab94b67a5f08f21de16ed3d/users/8ca0bff7c64c437d9ce2004d264fcf33/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:24.444076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 984/1963] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:24.449210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.450507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.450800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.451079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.493082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] MFA Rules not processed for user `8ca0bff7c64c437d9ce2004d264fcf33`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:24.496001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.531764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b3312-b903-4022-bad1-8db3d4705172 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.537183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.538017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.538403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.538746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.539507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.541641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.573748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 980/1964] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:24 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:24.573748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.575796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.581427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 985/1965] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:24.584802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.600882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9af945-4a7f-4700-abb9-c63cf1437a6f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.637115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:24.656079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cbcea-ad3e-4007-b598-6a6bc23a6fb9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.662986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.663964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.664238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.664511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] PATH_INFO: `/v3/projects/7a9f17d6ec6a4ecb93286597f8ae56e5` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.665702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9f17d6ec6a4ecb93286597f8ae56e5)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.667761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.668738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.669422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.669702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.670117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.670815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.672758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.697251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 986/1966] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:24 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:24.698658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.700519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.706912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:24.724525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9f17d6ec6a4ecb93286597f8ae56e5', 'request_id': 'req-408a113f-97d2-45f2-be91-4b169fd4e7b6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.731630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9f17d6ec6a4ecb93286597f8ae56e5', 'request_id': 'req-408a113f-97d2-45f2-be91-4b169fd4e7b6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.739210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9f17d6ec6a4ecb93286597f8ae56e5', 'request_id': 'req-408a113f-97d2-45f2-be91-4b169fd4e7b6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.753520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] The token cache is being invalidate because project 7a9f17d6ec6a4ecb93286597f8ae56e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:24.753884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.754548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.756401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 981/1967] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:24 2022] DELETE /identity/v3/projects/7a9f17d6ec6a4ecb93286597f8ae56e5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:24.759810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.761257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.763133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007871e-5996-4164-bbdf-47eea71d89e4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.818708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.819643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.819971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.820574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.823629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.849166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a113f-97d2-45f2-be91-4b169fd4e7b6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.854187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.854915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.855224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.855491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.856258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.857889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.868477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 982/1968] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:24.869377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.871206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.883856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109df0b-a17f-479f-9a25-693cb7c39d96 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.889213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.890079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.890393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.890675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] PATH_INFO: `/v3/users/743fffd50e714a16b9d7dfe4609999d0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.892039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743fffd50e714a16b9d7dfe4609999d0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.893971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.895646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.897162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.908398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 987/1969] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:24.912956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.925521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28e9f1-56b2-4ad3-9774-4c24a38fe944 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:24.932143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:24.933770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:24.934177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:24.934817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] PATH_INFO: `/v3/projects/573d9344a8e640cdbd27da4533fbd85b/users/4e9ef7cca6334b48a596c66d90827df3/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:24.950606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743fffd50e714a16b9d7dfe4609999d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.951563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743fffd50e714a16b9d7dfe4609999d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.957515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573d9344a8e640cdbd27da4533fbd85b, user_id=4e9ef7cca6334b48a596c66d90827df3, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:24.962830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:24.963209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743fffd50e714a16b9d7dfe4609999d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.964662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743fffd50e714a16b9d7dfe4609999d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.978816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743fffd50e714a16b9d7dfe4609999d0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:24.981117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 988/1970] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:24 2022] PUT /identity/v3/projects/573d9344a8e640cdbd27da4533fbd85b/users/4e9ef7cca6334b48a596c66d90827df3/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:24.982012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:24.983685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:24.989540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:25.006992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 983/1971] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:24 2022] DELETE /identity/v3/users/743fffd50e714a16b9d7dfe4609999d0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:25.023003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:25.024956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.040509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:25.050134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd849a4-aae5-47f0-8ccc-747aadb449a1 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.113570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b7e8-ae09-4df9-b3b8-fd7d7e59d550 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.126935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.128000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.128193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.128528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.129624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.131695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.144362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 984/1972] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:25.145854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.146581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.146948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.147274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.148515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:25.150136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.159844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946dd0e-f94c-441b-bf2d-806e6a71fd85 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.165227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260de5-f470-42f5-b01b-a4c114211a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.166154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260de5-f470-42f5-b01b-a4c114211a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.166459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260de5-f470-42f5-b01b-a4c114211a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.166740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260de5-f470-42f5-b01b-a4c114211a61 admin admin] PATH_INFO: `/v3/projects/573d9344a8e640cdbd27da4533fbd85b/users/4e9ef7cca6334b48a596c66d90827df3/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.169059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e260de5-f470-42f5-b01b-a4c114211a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573d9344a8e640cdbd27da4533fbd85b, user_id=4e9ef7cca6334b48a596c66d90827df3, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.171769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e260de5-f470-42f5-b01b-a4c114211a61 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.185313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 985/1973] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:25 2022] PUT /identity/v3/projects/573d9344a8e640cdbd27da4533fbd85b/users/4e9ef7cca6334b48a596c66d90827df3/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:25.197301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.197641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.197968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.236934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] MFA Rules not processed for user `4e9ef7cca6334b48a596c66d90827df3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:25.239992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.242985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.244781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.258165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 989/1974] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:25.357714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 986/1975] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:25.365602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:25.368195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.395960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d197d-16c7-4d3c-ad51-5e76b74ae930 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.460897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:25.463488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.475150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.476107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.476428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.476748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] PATH_INFO: `/v3/projects/009e89806cce42409aad807d5d8838b2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.478311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009e89806cce42409aad807d5d8838b2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.480387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.544941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009e89806cce42409aad807d5d8838b2', 'request_id': 'req-72e3919f-988f-4e98-bddc-ea317097a1d4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:25.552435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fbfdc-d82b-481c-8a0d-3cc76b3832a9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.553010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009e89806cce42409aad807d5d8838b2', 'request_id': 'req-72e3919f-988f-4e98-bddc-ea317097a1d4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:25.554782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:25.565078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009e89806cce42409aad807d5d8838b2', 'request_id': 'req-72e3919f-988f-4e98-bddc-ea317097a1d4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:25.576358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:25.580461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] The token cache is being invalidate because project 009e89806cce42409aad807d5d8838b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:25.581019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:25.581724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72e3919f-988f-4e98-bddc-ea317097a1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:25.583290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 990/1976] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:25 2022] DELETE /identity/v3/projects/009e89806cce42409aad807d5d8838b2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:25.589223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9db15-aae6-446d-bc14-37c2a6363ec1 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.589586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9db15-aae6-446d-bc14-37c2a6363ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.590274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9db15-aae6-446d-bc14-37c2a6363ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.612560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c9f9db15-aae6-446d-bc14-37c2a6363ec1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:25.630293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c9f9db15-aae6-446d-bc14-37c2a6363ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.656873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.657904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.658329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.658744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.666618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.761985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 991/1977] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:25.773162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.773439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.773856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.789784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.792106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.796324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:25.797897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:25.810351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 987/1978] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:25.813756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 992/1979] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:25.819288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:25.857372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7d0a-1270-46a7-ba9f-75ddb024a07d service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.872822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.873740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.874335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.875916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.878677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:25.879650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.882899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.906617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 988/1980] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:25.908497 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:25.919895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cdf74-675a-404b-b291-ceec7e0a9d0f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:25.926109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:25.926843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:25.927244 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:25.927543 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:25.928363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:25.930515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:25.991591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 993/1981] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:25 2022] POST /identity/v3/projects => generated 361 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:25.992910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:26.006425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc945cc6-b87b-43ca-9225-b8c191be00f4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:26.137503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:26.138336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:26.138692 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:26.139467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:26.144340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:26.146483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:26.183825 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 989/1982] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:25 2022] POST /identity/v3/users => generated 281 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:26.187925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:26.221787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3015-8451-4454-bd03-56e6ea1c508d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:26.229851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:26.230909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:26.231305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:26.231695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:26.232555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:26.243831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:26.267059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 994/1983] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:26.273272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:26.305530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d76ef-be03-4bb7-bb5c-e2abcf8027fb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:26.311675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a7380-740e-4ee5-88c7-ebb895abb082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:26.312382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a7380-740e-4ee5-88c7-ebb895abb082 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:26.312663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a7380-740e-4ee5-88c7-ebb895abb082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:26.312936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a7380-740e-4ee5-88c7-ebb895abb082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:26.313671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a7380-740e-4ee5-88c7-ebb895abb082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:26.315383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a7380-740e-4ee5-88c7-ebb895abb082 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:26.328006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 990/1984] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:26.332889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:26.346954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc226d7-6242-429e-8de2-ea05796bcc33 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:26.356920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:26.359736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:26.360216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:26.360664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] PATH_INFO: `/v3/projects/7a363c00d4c64b2f9351dc55f98af76e/users/6012425ca9ac491890d2ab2cc323f328/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:26.387710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a363c00d4c64b2f9351dc55f98af76e, user_id=6012425ca9ac491890d2ab2cc323f328, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:26.392996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:26.415969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 995/1985] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:26 2022] PUT /identity/v3/projects/7a363c00d4c64b2f9351dc55f98af76e/users/6012425ca9ac491890d2ab2cc323f328/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:26.428789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:26.428789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:26.428789 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:26.473423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] MFA Rules not processed for user `6012425ca9ac491890d2ab2cc323f328`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:26.474970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:26.507852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:26.590556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 991/1986] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:26.940818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:26.942995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:27.034231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:27.035617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:27.035902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78faf0a8-5be8-4e66-95d6-9d46a3fbc893 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:27.039733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:27.040427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.040781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.040985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.053304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:27.054759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:27.068708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 996/1987] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:27.085319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:27.126658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:27.162687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754e863-71d3-4fed-92b4-3bb53813a202 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:27.168709 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:27.169547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.169830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.170105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] PATH_INFO: `/v3/users/bf363ab070394b04b49ed3abd97d48c0` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.171885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf363ab070394b04b49ed3abd97d48c0)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:27.174935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:27.177816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da9a5c-99eb-401e-b927-3aa132bdaee5 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:27.184519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:27.185563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.185914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.186253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] PATH_INFO: `/v3/users/6012425ca9ac491890d2ab2cc323f328` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.187563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6012425ca9ac491890d2ab2cc323f328)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:27.189952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:27.249357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf363ab070394b04b49ed3abd97d48c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.250581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf363ab070394b04b49ed3abd97d48c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.263829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6012425ca9ac491890d2ab2cc323f328'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.264700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf363ab070394b04b49ed3abd97d48c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.264854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6012425ca9ac491890d2ab2cc323f328'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.266386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6012425ca9ac491890d2ab2cc323f328'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.269148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf363ab070394b04b49ed3abd97d48c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.296901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcbaf00e-7da0-4490-ae6f-aef45c0c4a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf363ab070394b04b49ed3abd97d48c0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.324827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6012425ca9ac491890d2ab2cc323f328'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.328091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 992/1988] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:27 2022] DELETE /identity/v3/users/bf363ab070394b04b49ed3abd97d48c0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:27.332713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.333004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.333284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.351002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-74101727-690a-4bbe-b69f-2faed8bd1bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6012425ca9ac491890d2ab2cc323f328'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:27.368846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:27.371225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:27.381339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 997/1989] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:27 2022] DELETE /identity/v3/users/6012425ca9ac491890d2ab2cc323f328 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:27.386670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.386957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.387230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.433872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:27.435111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:27.454258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:27.497992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 993/1990] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:27.520112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:27.523139 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:27.542782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 998/1991] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:27.574638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:27.576643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:27.600628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740b986-4029-4394-ac66-97a7ead96740 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:27.662887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:27.671436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c29520-e8bc-4409-b398-9d501a7d518d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:27.676617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:27.677099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:27.677682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.677792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:27.678025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.678301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.678418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:27.678601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:27.689271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:27.691218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:27.692196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:27.693648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:27.704363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 994/1992] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:27.707486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 999/1993] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:28.448666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:28.450226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:28.477504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7db3d-d591-4c50-9814-0890ecbd06b6 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:28.482728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:28.483609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:28.483911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:28.484197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] PATH_INFO: `/v3/projects/5fa779639da44c10aa263e2780f8e6bf` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:28.487443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa779639da44c10aa263e2780f8e6bf)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:28.489406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:28.609640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:28.610952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:28.628337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa779639da44c10aa263e2780f8e6bf', 'request_id': 'req-e6e52f41-deb9-433e-8fc9-20a13e41c51f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.636058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525fac-8228-446e-9c77-4afb6717a6ef service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:28.641651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa779639da44c10aa263e2780f8e6bf', 'request_id': 'req-e6e52f41-deb9-433e-8fc9-20a13e41c51f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.657425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa779639da44c10aa263e2780f8e6bf', 'request_id': 'req-e6e52f41-deb9-433e-8fc9-20a13e41c51f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.684019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] The token cache is being invalidate because project 5fa779639da44c10aa263e2780f8e6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:28.684554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.686196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.688574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 995/1994] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:28 2022] DELETE /identity/v3/projects/5fa779639da44c10aa263e2780f8e6bf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:28.693306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:28.696472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:28.752147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:28.780431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:28.781509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:28.781928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:28.782339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] PATH_INFO: `/v3/users/2c088c79bbf4444a981d82dbd8a4b935` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:28.791634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c088c79bbf4444a981d82dbd8a4b935)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:28.793816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:28.849498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e52f41-deb9-433e-8fc9-20a13e41c51f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:28.880636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:28.881555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:28.882965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:28.883241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] PATH_INFO: `/v3/users/4e9ef7cca6334b48a596c66d90827df3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:28.884619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9ef7cca6334b48a596c66d90827df3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:28.893799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:28.915346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c088c79bbf4444a981d82dbd8a4b935'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.916003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c088c79bbf4444a981d82dbd8a4b935'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.916677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c088c79bbf4444a981d82dbd8a4b935'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.932267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c088c79bbf4444a981d82dbd8a4b935'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.954407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c088c79bbf4444a981d82dbd8a4b935'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:28.973431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1000/1995] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:28 2022] DELETE /identity/v3/users/2c088c79bbf4444a981d82dbd8a4b935 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:28.974319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:28.975788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.003303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9ef7cca6334b48a596c66d90827df3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.006824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9ef7cca6334b48a596c66d90827df3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.022086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9ef7cca6334b48a596c66d90827df3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.022536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9ef7cca6334b48a596c66d90827df3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.047489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a2e6ae5d-a6ff-4c19-9a97-de4b3873e119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9ef7cca6334b48a596c66d90827df3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.077980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 996/1996] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:28 2022] DELETE /identity/v3/users/4e9ef7cca6334b48a596c66d90827df3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:29.079363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.079651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.079972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.110884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:29.112106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.148286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cd37c-dd23-4ea9-95ef-775fdd7ef4f0 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:29.177638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:29.241301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 997/1997] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:29.266692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:29.267574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.267852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.268619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] PATH_INFO: `/v3/projects/7a363c00d4c64b2f9351dc55f98af76e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.270559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a363c00d4c64b2f9351dc55f98af76e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:29.272729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:29.302553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:29.304740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.373582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a363c00d4c64b2f9351dc55f98af76e', 'request_id': 'req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.393989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:29.394612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a363c00d4c64b2f9351dc55f98af76e', 'request_id': 'req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.410040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a363c00d4c64b2f9351dc55f98af76e', 'request_id': 'req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.421931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ce2c7-ebdf-46de-9e1c-3fd011fb2e33 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:29.428662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] The token cache is being invalidate because project 7a363c00d4c64b2f9351dc55f98af76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:29.428806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.430605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-64f14905-ba2e-4cf3-8dc9-9d220e3d5d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:29.435446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1001/1998] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:28 2022] DELETE /identity/v3/projects/7a363c00d4c64b2f9351dc55f98af76e => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:29.440683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b27d8d-2d9b-4b4a-b2e0-2c54c2198e7e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.440979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b27d8d-2d9b-4b4a-b2e0-2c54c2198e7e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.441252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b27d8d-2d9b-4b4a-b2e0-2c54c2198e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.483635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-07b27d8d-2d9b-4b4a-b2e0-2c54c2198e7e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:29.484940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-07b27d8d-2d9b-4b4a-b2e0-2c54c2198e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.507688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:29.509113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.511060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.511903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.516644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.626099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1002/1999] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:29.633096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.633607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.633975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.658520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:29.661912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:29.671969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:29.673306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.685052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 998/2000] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:29.689817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1003/2001] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:29.694069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:29.704654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e8065-bd2e-4d58-95a8-21a4838197b2 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:29.709667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:29.710449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.710795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.711133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.712012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:29.712384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:29.714232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:29.729260 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 999/2002] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:01:29.733690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:29.744791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c48de-92a7-4738-bedd-a32e8e699b02 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:29.749830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:29.750576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:29.750940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:29.751275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:29.752156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:29.754144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:29.797038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1004/2003] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:29 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:29.801723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:29.821233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7842-8f93-4c0f-abee-c9502df9e20f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:29.866437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:29.868466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:29.936071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:30.004656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.005355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.005637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.005913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.006633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.013564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.060797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1000/2004] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:29 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:30.061806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.069612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f3e02-9357-4431-825c-e7334081f0fd admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.081416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.083533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.084803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690addf-e5d8-4a1b-a8fe-7bafc91bb01c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.085365 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.085717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] PATH_INFO: `/v3/projects/573d9344a8e640cdbd27da4533fbd85b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.087483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=573d9344a8e640cdbd27da4533fbd85b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.092779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.102415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.103080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.103424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.104063 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.104814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.106447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.134460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1001/2005] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:30.135680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.161474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94f571-7338-4b6f-bd01-58781bcb22e6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.181971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573d9344a8e640cdbd27da4533fbd85b', 'request_id': 'req-3df2e693-e70b-4b54-a4de-19ca3bad10f9'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.186206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:30.192951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573d9344a8e640cdbd27da4533fbd85b', 'request_id': 'req-3df2e693-e70b-4b54-a4de-19ca3bad10f9'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.208659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573d9344a8e640cdbd27da4533fbd85b', 'request_id': 'req-3df2e693-e70b-4b54-a4de-19ca3bad10f9'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.228470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] The token cache is being invalidate because project 573d9344a8e640cdbd27da4533fbd85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:30.228871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.230686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3df2e693-e70b-4b54-a4de-19ca3bad10f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.234337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1005/2006] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:29 2022] DELETE /identity/v3/projects/573d9344a8e640cdbd27da4533fbd85b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:30.239067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.239404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.239691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.276451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:30.277558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.295446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.296099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.296380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.296649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.297390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.299148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.328920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1002/2007] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:30.329906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.339890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.361306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b4b0-5d84-427a-b9df-d04f8ee71bcf admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.390131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e322ee-5d3d-4e05-b395-cf1a34d45ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.391058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e322ee-5d3d-4e05-b395-cf1a34d45ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.391344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e322ee-5d3d-4e05-b395-cf1a34d45ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.391759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e322ee-5d3d-4e05-b395-cf1a34d45ba4 admin admin] PATH_INFO: `/v3/projects/414f8213431344ce93e298181f5572b6/users/66a9dfdc845c4a14b0f4e4b536722360/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.417308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e322ee-5d3d-4e05-b395-cf1a34d45ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414f8213431344ce93e298181f5572b6, user_id=66a9dfdc845c4a14b0f4e4b536722360, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.421759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e322ee-5d3d-4e05-b395-cf1a34d45ba4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.433170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1006/2008] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:30.433816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.441826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.476757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1003/2009] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:30 2022] PUT /identity/v3/projects/414f8213431344ce93e298181f5572b6/users/66a9dfdc845c4a14b0f4e4b536722360/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:30.476757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6e0b2-9a8d-4988-a0d1-4ca0586f6a8f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.476757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6e0b2-9a8d-4988-a0d1-4ca0586f6a8f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.476757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6e0b2-9a8d-4988-a0d1-4ca0586f6a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.477169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf67576-f7c1-42a6-89bb-5f3d5372118c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.498173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dda6e0b2-9a8d-4988-a0d1-4ca0586f6a8f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:30.499404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dda6e0b2-9a8d-4988-a0d1-4ca0586f6a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.561708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:30.587357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.588812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.589010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.589286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] PATH_INFO: `/v3/projects/4ee0735ab2cf469c8662621a6740ca9e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.594002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee0735ab2cf469c8662621a6740ca9e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.598212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.626766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1004/2010] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:30.630625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.630912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.631579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.713165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] MFA Rules not processed for user `66a9dfdc845c4a14b0f4e4b536722360`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:30.717693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.728370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee0735ab2cf469c8662621a6740ca9e', 'request_id': 'req-72fb9291-55fa-4e6f-a58d-1f49bd03444b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.735572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee0735ab2cf469c8662621a6740ca9e', 'request_id': 'req-72fb9291-55fa-4e6f-a58d-1f49bd03444b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.757038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee0735ab2cf469c8662621a6740ca9e', 'request_id': 'req-72fb9291-55fa-4e6f-a58d-1f49bd03444b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.779121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] The token cache is being invalidate because project 4ee0735ab2cf469c8662621a6740ca9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:30.779826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.780981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:30.789405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1007/2011] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:30 2022] DELETE /identity/v3/projects/4ee0735ab2cf469c8662621a6740ca9e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:30.790713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.792851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.875211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb9291-55fa-4e6f-a58d-1f49bd03444b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.880451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.881205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.881553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.881896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.882719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:30.883355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.885755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.891023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1005/2012] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:30.893129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.897863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.899589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1008/2013] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:30.905090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:30.909190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:30.931384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbbe12-a27e-493d-8671-a45e0b0b96d4 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.945959 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.947316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.947812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.948438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] PATH_INFO: `/v3/users/8ca0bff7c64c437d9ce2004d264fcf33` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.950099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca0bff7c64c437d9ce2004d264fcf33)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:30.953092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:30.972434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de18388-4b29-4c13-b58d-22a85323a31c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:30.976247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:30.977054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:30.977623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:30.978016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:30.997090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:30.998686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.000839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.028021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1009/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:31.030236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.060209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e17832-5040-4113-9964-391b96d55f26 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.074774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca0bff7c64c437d9ce2004d264fcf33'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.075430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.075658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca0bff7c64c437d9ce2004d264fcf33'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.076189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.076476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.076835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.077660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.079571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.087509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca0bff7c64c437d9ce2004d264fcf33'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.088499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca0bff7c64c437d9ce2004d264fcf33'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.105250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca0bff7c64c437d9ce2004d264fcf33'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.110356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:31.121131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1006/2015] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:30 2022] DELETE /identity/v3/users/8ca0bff7c64c437d9ce2004d264fcf33 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:31.152228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1010/2016] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:31 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:31.153854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.156578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.283571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a4840-02d2-44fc-a910-1b13e36fb7e6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.380248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.381201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.381614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.382156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.383027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.385712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.421846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1007/2017] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:31 2022] POST /identity/v3/users => generated 292 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:31.424717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.438618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd240ebc-43ed-454e-8904-af7991494462 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.448831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.449942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.450392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.451066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.452292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.454276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.473972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1011/2018] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:31.475019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.490658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc067-f8de-4769-b932-9ac9519b843f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.497823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.498279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.498732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.499127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.500113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.501926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.516603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1008/2019] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:31.521020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.534542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d04ede-2965-4f1d-afbc-1b3d0eaea867 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.544404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb26071-002d-4ce2-8eb4-00003c4e2c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.547295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26071-002d-4ce2-8eb4-00003c4e2c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.547764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26071-002d-4ce2-8eb4-00003c4e2c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.548151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26071-002d-4ce2-8eb4-00003c4e2c4a admin admin] PATH_INFO: `/v3/projects/4b31b208d8b7428fb271231514de2a31/users/f4ae78fab9504d41896d7bfb1f9f60cc/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.574382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb26071-002d-4ce2-8eb4-00003c4e2c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b31b208d8b7428fb271231514de2a31, user_id=f4ae78fab9504d41896d7bfb1f9f60cc, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.581818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb26071-002d-4ce2-8eb4-00003c4e2c4a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.611066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1012/2020] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:31 2022] PUT /identity/v3/projects/4b31b208d8b7428fb271231514de2a31/users/f4ae78fab9504d41896d7bfb1f9f60cc/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:31.612682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.614430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.629060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.629413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.629738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.652220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b1ec5-8eb8-4320-a1b1-bd18aefa3262 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.668116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:31.671110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] MFA Rules not processed for user `f4ae78fab9504d41896d7bfb1f9f60cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:31.672755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.726485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.727427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.727713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.728027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] PATH_INFO: `/v3/projects/fb5540263ab94b67a5f08f21de16ed3d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.729741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5540263ab94b67a5f08f21de16ed3d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.731758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.772196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1013/2021] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:31.788473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:31.789972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.815803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5540263ab94b67a5f08f21de16ed3d', 'request_id': 'req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.823746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5540263ab94b67a5f08f21de16ed3d', 'request_id': 'req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.830371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5540263ab94b67a5f08f21de16ed3d', 'request_id': 'req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.845763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] The token cache is being invalidate because project fb5540263ab94b67a5f08f21de16ed3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:31.846045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.846668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6dd9699c-a6a3-4d9c-bc72-cfa05763974f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:31.848251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1009/2022] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:31 2022] DELETE /identity/v3/projects/fb5540263ab94b67a5f08f21de16ed3d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:31.852899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff15ecd-847f-4346-942c-1ae2f2cb27d8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.853196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff15ecd-847f-4346-942c-1ae2f2cb27d8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.853570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff15ecd-847f-4346-942c-1ae2f2cb27d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.857097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaa6f8-a7b9-4fde-b53a-d314b98b5fe3 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:31.872107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8ff15ecd-847f-4346-942c-1ae2f2cb27d8 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:31.873299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8ff15ecd-847f-4346-942c-1ae2f2cb27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.908246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:31.909003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.909352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.909688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.911597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.933250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:31.948977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1010/2023] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:31.957284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299556-b348-43a7-8b88-a9edb409923e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:31.957568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299556-b348-43a7-8b88-a9edb409923e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:31.957840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299556-b348-43a7-8b88-a9edb409923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:31.978934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-af299556-b348-43a7-8b88-a9edb409923e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:31.980100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-af299556-b348-43a7-8b88-a9edb409923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:31.989473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1011/2024] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:31.992956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:31.994570 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:31.996993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af299556-b348-43a7-8b88-a9edb409923e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.006807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af299556-b348-43a7-8b88-a9edb409923e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.007099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1014/2025] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:32.011757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.012455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.012745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.013024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.013763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:32.014073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.015845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.033094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.034507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.039625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1012/2026] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:32.040641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.049650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb5d24-b35e-4d4d-b52a-9531174f0f84 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.051448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809eba0-bc29-4ecb-a3ab-350843b22e0b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.057207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.057599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.058089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.058275 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.058381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.058544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.058647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] PATH_INFO: `/v3/users/f4ae78fab9504d41896d7bfb1f9f60cc` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.058824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.059538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.059899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ae78fab9504d41896d7bfb1f9f60cc)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.061610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.061927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.091932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1013/2027] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:32.094960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.101340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:32.120834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ae78fab9504d41896d7bfb1f9f60cc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.121998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ae78fab9504d41896d7bfb1f9f60cc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.123721 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ae78fab9504d41896d7bfb1f9f60cc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.136613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ae78fab9504d41896d7bfb1f9f60cc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.156738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-53cdd9df-3773-4405-a68b-97b462cb0f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ae78fab9504d41896d7bfb1f9f60cc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.170856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab48a5-601b-4fcf-af85-d64170a41e3c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.172732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1015/2028] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:32 2022] DELETE /identity/v3/users/f4ae78fab9504d41896d7bfb1f9f60cc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:32.176895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.177181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.177479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.197117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:32.198467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.245406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.246086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.246364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.246637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.247600 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.249553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.260369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1016/2029] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:32.261227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.262446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.274703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b914b06-e209-41a2-a57e-938a53af942d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.277054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1014/2030] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/users => generated 304 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:32.277798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.279407 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.282952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.283852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.284206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.284485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] PATH_INFO: `/v3/users/51d401bbf09b4dc7923a1826a976ff17` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.285688 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d401bbf09b4dc7923a1826a976ff17)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.287855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.318665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:32.340295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d401bbf09b4dc7923a1826a976ff17'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.341064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d401bbf09b4dc7923a1826a976ff17'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.341175 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f6bdf-b13c-461a-a57e-262111820057 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.341829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d401bbf09b4dc7923a1826a976ff17'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.344807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.345467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.345754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.346030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.348121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.350701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d401bbf09b4dc7923a1826a976ff17'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.361026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.362468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.363263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-58eda77e-4e13-4dd0-bd3d-497f7ea8b50f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d401bbf09b4dc7923a1826a976ff17'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.372192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1015/2031] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:32.373047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.374357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.375733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1017/2032] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:32 2022] DELETE /identity/v3/users/51d401bbf09b4dc7923a1826a976ff17 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:32.379846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.380162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.380430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.386387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fa674-0bd6-4657-92a5-4e9cbd490b50 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.398192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:32.399393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.436339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:32.467616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.468334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.468617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.468895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.470580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.472247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.484584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1016/2033] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:32.488347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.494922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1018/2034] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:32.499464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320256e4-3739-41e5-a687-6e3da67d43e3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.504332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.505003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.505284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.505557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.506026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.506290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.507778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.507931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.518949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1017/2035] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:32.522804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.533225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c26b5c-9335-4543-8356-c92424d5ed4d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.538631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9845b-76d0-4975-b5fe-af3d974eb891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.539511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9845b-76d0-4975-b5fe-af3d974eb891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.539786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9845b-76d0-4975-b5fe-af3d974eb891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.540092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9845b-76d0-4975-b5fe-af3d974eb891 admin admin] PATH_INFO: `/v3/projects/e223f3f583a443bf993d7112ae8c2d92/users/4498ef51257044f3aee634437ccb9b0f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.558507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9845b-76d0-4975-b5fe-af3d974eb891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e223f3f583a443bf993d7112ae8c2d92, user_id=4498ef51257044f3aee634437ccb9b0f, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.561289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9845b-76d0-4975-b5fe-af3d974eb891 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.584770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1018/2036] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:32 2022] PUT /identity/v3/projects/e223f3f583a443bf993d7112ae8c2d92/users/4498ef51257044f3aee634437ccb9b0f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:32.587651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812fc6d-4155-4d19-ab9f-cca13e02c347 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.597887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.598333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.598611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.666647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] MFA Rules not processed for user `4498ef51257044f3aee634437ccb9b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:32.666647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.671596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.672295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.672575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.672851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.688646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:32.761601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1019/2037] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:32.762769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.764922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.766157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.767815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.777592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1019/2038] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:32.778416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.782495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f7c3f-08a1-47d4-8e4f-7f66d1b40e86 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.787986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.789093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.789446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.789572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f668801-b058-4764-8012-2c70465fca7f service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.789788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] PATH_INFO: `/v3/projects/4b31b208d8b7428fb271231514de2a31` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.791088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b31b208d8b7428fb271231514de2a31)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.793331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.795695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:32.796427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.796662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.796936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.798652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:32.800625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:32.853010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1020/2039] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/projects => generated 407 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:32.859128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:32.871134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b31b208d8b7428fb271231514de2a31', 'request_id': 'req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.877536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b31b208d8b7428fb271231514de2a31', 'request_id': 'req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.887201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b31b208d8b7428fb271231514de2a31', 'request_id': 'req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.903587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] The token cache is being invalidate because project 4b31b208d8b7428fb271231514de2a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:32.904101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.905283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-161609b6-e1ff-4702-a1dc-3ab5faac7bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:32.907440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1020/2040] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:32 2022] DELETE /identity/v3/projects/4b31b208d8b7428fb271231514de2a31 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:32.912572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c3599-89a4-4868-9402-2e8a60c03634 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:32.913095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c3599-89a4-4868-9402-2e8a60c03634 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:32.913353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c3599-89a4-4868-9402-2e8a60c03634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:32.951242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-268c3599-89a4-4868-9402-2e8a60c03634 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:32.954429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-268c3599-89a4-4868-9402-2e8a60c03634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:32.966817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f695-4814-47d7-97f8-0be289a2b2ba admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:32.978234 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:33.043930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1021/2041] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:33.050196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.050727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.051011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.051140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.051761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.052080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.052360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.053063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.055035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.074972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:33.076244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.081586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1021/2042] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:32 2022] POST /identity/v3/users => generated 310 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:33.085018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.087075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.088577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1022/2043] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:33.092957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.102699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138693c-918c-4357-9dc6-9daecdede033 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.104048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a134d69-13af-410e-8fac-cb569ac6683a None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.108131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.108969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.109105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.109301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.109529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.109745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.110003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.110319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.110430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.111155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:33.111461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.112137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.113110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.126009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1023/2045] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:33.126202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1022/2045] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:33.126889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.132990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.137202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.145440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0efcba-3cc3-4951-8439-a16485123611 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.150068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af999ddb-1f45-4cec-866d-8cca94542495 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.152043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.153565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.153742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.154022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] PATH_INFO: `/v3/projects/29ee9cc161ab4fb7ab39c606d1520a77/users/5025eac87a8e4ec88ec94c4606d65b97/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.156195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.157223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.157506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.157781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] PATH_INFO: `/v3/projects/fcfc1612a14d4b13af7da45fd6171fe4` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.158951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfc1612a14d4b13af7da45fd6171fe4)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.160937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.165972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:33.171848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ee9cc161ab4fb7ab39c606d1520a77, user_id=5025eac87a8e4ec88ec94c4606d65b97, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.175587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.189349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1023/2046] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:33 2022] PUT /identity/v3/projects/29ee9cc161ab4fb7ab39c606d1520a77/users/5025eac87a8e4ec88ec94c4606d65b97/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:33.190151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.211622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfc1612a14d4b13af7da45fd6171fe4', 'request_id': 'req-f8d86ae2-574b-42e1-8eb6-1803a93293c9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.217806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfc1612a14d4b13af7da45fd6171fe4', 'request_id': 'req-f8d86ae2-574b-42e1-8eb6-1803a93293c9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.224608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfc1612a14d4b13af7da45fd6171fe4', 'request_id': 'req-f8d86ae2-574b-42e1-8eb6-1803a93293c9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.239197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] The token cache is being invalidate because project fcfc1612a14d4b13af7da45fd6171fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:33.239490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.240182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.241794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1024/2047] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:33 2022] DELETE /identity/v3/projects/fcfc1612a14d4b13af7da45fd6171fe4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:33.242626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.243927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.258704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a061ef-95a4-4016-98d9-34bf4b263220 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.310718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d86ae2-574b-42e1-8eb6-1803a93293c9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.315329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.316029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.316309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.316583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.316888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.317316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.317553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.317828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.318097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.318812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.318917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.321069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.329553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1025/2048] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:33.330393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.332066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.344687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690bb6-cc65-497f-9df7-14576b65d821 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.349228 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.350083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.350348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1024/2049] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:33 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:33.351144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.353632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.353936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] PATH_INFO: `/v3/users/117df7b26d1848078a0881df7c889c51` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.355332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=117df7b26d1848078a0881df7c889c51)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.357301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.369151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:33.409173 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '117df7b26d1848078a0881df7c889c51'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.409875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '117df7b26d1848078a0881df7c889c51'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.413221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b4791-1595-45df-8441-3c9deed66234 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.417957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '117df7b26d1848078a0881df7c889c51'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.418582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '117df7b26d1848078a0881df7c889c51'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.418710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.419538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.419829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.420160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] PATH_INFO: `/v3/projects/29ee9cc161ab4fb7ab39c606d1520a77/users/5025eac87a8e4ec88ec94c4606d65b97/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.422084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ee9cc161ab4fb7ab39c606d1520a77, user_id=5025eac87a8e4ec88ec94c4606d65b97, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.424758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.430657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bbb73ed8-25c3-466d-a1a8-8d78d06ad0e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '117df7b26d1848078a0881df7c889c51'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:33.442564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1025/2050] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:33 2022] PUT /identity/v3/projects/29ee9cc161ab4fb7ab39c606d1520a77/users/5025eac87a8e4ec88ec94c4606d65b97/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:33.442564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.443237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1026/2051] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:33 2022] DELETE /identity/v3/users/117df7b26d1848078a0881df7c889c51 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:33.445602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.447627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.447963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.448223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.465218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:33.481276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] MFA Rules not processed for user `5025eac87a8e4ec88ec94c4606d65b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:33.482777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.520784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526c126-17f1-4c5a-892b-06a327052b38 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.559523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1027/2052] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:33.571127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.573093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.591233 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.592331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.593128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.593389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.594187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.597300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.630350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1026/2053] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:33 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:33.634755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.637685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7ef2e-fdc8-4854-b1b6-927ca1542587 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.641812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.642565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.642760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.643465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.645691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadaae3-4c50-433d-90ae-870722fdd0c8 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.651074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.651739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.652051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.652446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.653229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.653401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.654857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.654974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.665676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1028/2054] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:33.671772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.682076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1027/2055] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:33.687434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b70a-b93a-4310-afc8-e7c6ce252179 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.694490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.695981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.696267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.696545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.698064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.700120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.719734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1029/2056] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:33.720916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.735678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e792f-25ec-471f-add1-ac05e1d297e6 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.740629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994ab38-6f54-4ad1-837c-9f396d048b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.741541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994ab38-6f54-4ad1-837c-9f396d048b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.741837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994ab38-6f54-4ad1-837c-9f396d048b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.742100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994ab38-6f54-4ad1-837c-9f396d048b2b admin admin] PATH_INFO: `/v3/projects/b3da500196ac410abce439fc4988f9da/users/9a4af01953434898a651726cd206f27e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.759865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994ab38-6f54-4ad1-837c-9f396d048b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3da500196ac410abce439fc4988f9da, user_id=9a4af01953434898a651726cd206f27e, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.762814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994ab38-6f54-4ad1-837c-9f396d048b2b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.781239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.782503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1028/2057] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:33 2022] PUT /identity/v3/projects/b3da500196ac410abce439fc4988f9da/users/9a4af01953434898a651726cd206f27e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:33.784967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.793993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.794282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.794561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.824501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:33.830986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800d828-3411-47f8-8672-bfe9334a3143 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:33.840338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] MFA Rules not processed for user `9a4af01953434898a651726cd206f27e`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:33.842276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:33.933523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:33.935280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1029/2058] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:33.936213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:33.940250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:33.940539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:33.940818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] PATH_INFO: `/v3/projects/8cbda8c6a64d4f69b53887026f0d6221` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:33.942241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbda8c6a64d4f69b53887026f0d6221)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:33.944301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:33.960451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:33.998035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbda8c6a64d4f69b53887026f0d6221', 'request_id': 'req-61e6347c-6827-4f80-8e34-835024295be6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.004076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbda8c6a64d4f69b53887026f0d6221', 'request_id': 'req-61e6347c-6827-4f80-8e34-835024295be6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.011749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbda8c6a64d4f69b53887026f0d6221', 'request_id': 'req-61e6347c-6827-4f80-8e34-835024295be6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.014236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb582a-e8fc-4430-b963-23b5384c980c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.025006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] The token cache is being invalidate because project 8cbda8c6a64d4f69b53887026f0d6221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:34.025449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.026639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.028606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1030/2059] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:33 2022] DELETE /identity/v3/projects/8cbda8c6a64d4f69b53887026f0d6221 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:34.029386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.030667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.077098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.077717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.078001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.078276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.080219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.092747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6347c-6827-4f80-8e34-835024295be6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.096638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.097279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.097556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.097828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.099766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.144593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.146063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37a576-e766-43de-be68-8ab5e1b563b9 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.152572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:34.156325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1030/2060] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:34.157968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd7b2-a145-409f-805b-ad43ad657f32 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.157968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd7b2-a145-409f-805b-ad43ad657f32 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.158251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd7b2-a145-409f-805b-ad43ad657f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.175229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.177446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.178861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b36cd7b2-a145-409f-805b-ad43ad657f32 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:34.180122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b36cd7b2-a145-409f-805b-ad43ad657f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.189189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1031/2061] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:34.218401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.224860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.302241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1031/2062] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:34.313341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.313758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.314340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.335182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514eddfc-e730-466f-aa75-84b9944cb21c service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.336396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:34.337883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.349025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1032/2063] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:34.352588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.353682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.355446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.356200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.357337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] PATH_INFO: `/v3/users/9a4af01953434898a651726cd206f27e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.359047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4af01953434898a651726cd206f27e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.361246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.371103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00db669-f75d-4be4-ad2b-402fc1de7172 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.378760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.379646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.380052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.380381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.381662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:34.382357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.384172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.395421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:34.405919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1033/2064] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:34.410503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.423254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935074-e247-4b4a-9cdf-93015bfb541a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.427402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4af01953434898a651726cd206f27e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.428153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4af01953434898a651726cd206f27e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.430013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.430752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.431136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.431948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.433182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.434908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.443922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4af01953434898a651726cd206f27e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.445576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4af01953434898a651726cd206f27e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.465607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-76e70c09-cdd9-410c-ba56-c5023f67cbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4af01953434898a651726cd206f27e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:34.468408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1034/2065] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:34.473891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.475633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.493370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1032/2066] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:34 2022] DELETE /identity/v3/users/9a4af01953434898a651726cd206f27e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:34.493370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.493370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.493370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.514967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:34.516220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.588668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055af310-b266-4a8e-937d-ec7cc322061b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.594800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.595330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1033/2067] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:34.596576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.597008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.597412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.598296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.600806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.608479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.610118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.623333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1035/2068] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:34.624749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:34.628046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.640208 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b191b1-700a-4b14-8e39-d8a6bb7aa1eb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.647410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.648130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.648474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.648771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.649636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.651341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.663284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1036/2069] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:34.666293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.679207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125dc47-b6b7-4b12-b2ec-221d4cca8a51 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.681866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74216edb-efbe-4b1a-ae2f-e2a6fd2ada1f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.686026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1e22a-200b-4001-9f70-f6a22b1ff036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.686456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.687088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1e22a-200b-4001-9f70-f6a22b1ff036 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.687412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1e22a-200b-4001-9f70-f6a22b1ff036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.687504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.687684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1e22a-200b-4001-9f70-f6a22b1ff036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.687857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.688354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.689534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.692179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.698586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1e22a-200b-4001-9f70-f6a22b1ff036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.700176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1e22a-200b-4001-9f70-f6a22b1ff036 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.708836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1037/2070] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:34.710492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1034/2071] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:34.714744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.725179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d598b-b7fc-4315-95c8-9fbcd5be337e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.730894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.731809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.732184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.732470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] PATH_INFO: `/v3/projects/4e557c2302f5420496db480e0f2953cb/users/b9837681a5754ab881672e70bd1fe51d/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.749945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e557c2302f5420496db480e0f2953cb, user_id=b9837681a5754ab881672e70bd1fe51d, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.752848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:34.768340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1038/2072] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:34 2022] PUT /identity/v3/projects/4e557c2302f5420496db480e0f2953cb/users/b9837681a5754ab881672e70bd1fe51d/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:34.783115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.783510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.783838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.813454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] MFA Rules not processed for user `b9837681a5754ab881672e70bd1fe51d`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:34.814727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:34.888628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1035/2073] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:34.893602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:34.977274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e455f-58d3-433a-b5b0-2595e0340670 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:34.983144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:34.983932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:34.984223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:34.984503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:34.985206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:34.987150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.019956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.021555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.028088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1039/2074] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:34 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:35.032572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.043902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eff2a1-eca0-47fb-bce9-5a8615466e6d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.046020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:35.117488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.118370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.118652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.119084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] PATH_INFO: `/v3/projects/b3da500196ac410abce439fc4988f9da` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.120404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3da500196ac410abce439fc4988f9da)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.122323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.140617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50786de-acf4-46dd-8b21-97e163eb3c67 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.145277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.146041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.146382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.146715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.147473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.149510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.168287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:35.171223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1040/2075] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/users => generated 301 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:35.172747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3da500196ac410abce439fc4988f9da', 'request_id': 'req-81045d89-a63c-49bf-be64-71082abc8adc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.175388 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.178878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3da500196ac410abce439fc4988f9da', 'request_id': 'req-81045d89-a63c-49bf-be64-71082abc8adc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.185360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3da500196ac410abce439fc4988f9da', 'request_id': 'req-81045d89-a63c-49bf-be64-71082abc8adc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.185781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd93a87-4bef-45c0-83da-c8e66327240c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.199136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] The token cache is being invalidate because project b3da500196ac410abce439fc4988f9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:35.199429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.200108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81045d89-a63c-49bf-be64-71082abc8adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.201696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1036/2076] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:35 2022] DELETE /identity/v3/projects/b3da500196ac410abce439fc4988f9da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:35.206342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a5c98-dfd2-4ec1-a167-ba3339d9b868 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.206625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a5c98-dfd2-4ec1-a167-ba3339d9b868 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.206901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a5c98-dfd2-4ec1-a167-ba3339d9b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.224565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8e6a5c98-dfd2-4ec1-a167-ba3339d9b868 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:35.225752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8e6a5c98-dfd2-4ec1-a167-ba3339d9b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.243160 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.243832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.244193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.244475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.245330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.246954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.256884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1041/2077] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:35.261520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.263238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.277981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889fbbd-ca5e-4c55-8ecc-92209e8e4894 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.283374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.284436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.284715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.284982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] PATH_INFO: `/v3/projects/69b3e97931f14afb87f44d9fb6acd121/users/0463d4f8a3274081ae700cd1a3926928/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.299926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1037/2078] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:35.301980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.302267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.302550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.307750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b3e97931f14afb87f44d9fb6acd121, user_id=0463d4f8a3274081ae700cd1a3926928, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.311520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.321049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:35.322339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.325935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1042/2079] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:35 2022] PUT /identity/v3/projects/69b3e97931f14afb87f44d9fb6acd121/users/0463d4f8a3274081ae700cd1a3926928/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:35.329979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.383611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1038/2080] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:35.388312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.392263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca8a22-0192-463b-bb9f-a01c538b8847 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.397529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd472d9-6add-4b4d-bafc-73ea932ebd55 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.398919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.399597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.399897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.400205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.401058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.402485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.402717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.403183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.403462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.403730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.404527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:35.404870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.406409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.413259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1043/2081] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:35.417611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.423168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1039/2082] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:01:35.423309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:35.424115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.429459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d92b8a-7496-4d57-b56a-ba82ce50f559 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.434862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf69b09-bdca-4362-817f-f7e04e83f4de admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.435503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dbb7e-b8dc-499a-8cb1-e1d8e33ee479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.436380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dbb7e-b8dc-499a-8cb1-e1d8e33ee479 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.436657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dbb7e-b8dc-499a-8cb1-e1d8e33ee479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.436930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dbb7e-b8dc-499a-8cb1-e1d8e33ee479 admin admin] PATH_INFO: `/v3/projects/69b3e97931f14afb87f44d9fb6acd121/users/0463d4f8a3274081ae700cd1a3926928/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.438843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dbb7e-b8dc-499a-8cb1-e1d8e33ee479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b3e97931f14afb87f44d9fb6acd121, user_id=0463d4f8a3274081ae700cd1a3926928, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.439627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.440303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.440587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.440882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.441509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dbb7e-b8dc-499a-8cb1-e1d8e33ee479 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.441627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.443432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.452710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1044/2083] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:35 2022] PUT /identity/v3/projects/69b3e97931f14afb87f44d9fb6acd121/users/0463d4f8a3274081ae700cd1a3926928/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:35.463141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.463410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.463682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.472273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1040/2084] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:35.476349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.490409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] MFA Rules not processed for user `0463d4f8a3274081ae700cd1a3926928`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:35.491613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.517776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:35.539881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95565e-1dee-4dca-a74e-78c66b9eb61a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.544990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.545658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.545947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.546222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.546919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.548932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.561830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1045/2085] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:35.573267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1041/2086] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:35.574848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.578315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.580114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.585775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea8605-2be8-4fe7-a0ec-f7a47dcb9154 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.592171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.593127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.593471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.593821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.594599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.596296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.614466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1046/2087] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:35.614466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.621525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a02ce8-1cc2-48dd-8c37-278ac69d3169 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.626550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.627209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.627490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.627758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.628510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.630110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.638912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf496b-74d1-4d3c-94e2-c00703bd88dd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.644155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1047/2088] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:35.644297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c85344-9a82-49bf-a9f5-c28cfe78c4c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.644297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c85344-9a82-49bf-a9f5-c28cfe78c4c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.644297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c85344-9a82-49bf-a9f5-c28cfe78c4c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.644297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c85344-9a82-49bf-a9f5-c28cfe78c4c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.645207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.653589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c85344-9a82-49bf-a9f5-c28cfe78c4c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.655021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c85344-9a82-49bf-a9f5-c28cfe78c4c0 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.655156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab565-a876-44a8-9c65-74efb3db4a47 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.659833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.661023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.661321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.661632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] PATH_INFO: `/v3/projects/80a39c0a82e94755904019e1e805cfa2/users/71e626b7b8cd4d7fb761242e186fc898/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.664938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1042/2089] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:35.680920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a39c0a82e94755904019e1e805cfa2, user_id=71e626b7b8cd4d7fb761242e186fc898, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.683749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.697830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1048/2090] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:35 2022] PUT /identity/v3/projects/80a39c0a82e94755904019e1e805cfa2/users/71e626b7b8cd4d7fb761242e186fc898/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:35.707941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.708409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.708697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.737758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] MFA Rules not processed for user `71e626b7b8cd4d7fb761242e186fc898`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:35.739029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:35.797116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.812288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1043/2091] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:35.823575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.862554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3390889-f4f1-457f-8e80-e8eba30aa454 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.867658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.868568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.868860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.869352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] PATH_INFO: `/v3/users/b9837681a5754ab881672e70bd1fe51d` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.870747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9837681a5754ab881672e70bd1fe51d)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.872726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.877635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51a32-fd20-485a-9ba2-1472bd1c9e61 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.881703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.882358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.882642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.882924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.892501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.893853 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.903236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1044/2092] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:35.910500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:35.926144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9837681a5754ab881672e70bd1fe51d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.927000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9837681a5754ab881672e70bd1fe51d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.927783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9837681a5754ab881672e70bd1fe51d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.929894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:35.936720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:35.938341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9837681a5754ab881672e70bd1fe51d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.943073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc195c6-e163-483e-8a52-52a0a66d19b0 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:35.949264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:35.950199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.950541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.950890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] PATH_INFO: `/v3/users/71e626b7b8cd4d7fb761242e186fc898` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.952185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e626b7b8cd4d7fb761242e186fc898)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:35.952714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cd3870f4-9fb5-485e-af03-99f985d17d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9837681a5754ab881672e70bd1fe51d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:35.954221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:35.966051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1049/2093] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:35 2022] DELETE /identity/v3/users/b9837681a5754ab881672e70bd1fe51d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:35.970551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:35.970841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:35.971122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:35.990150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:35.991369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.005605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e626b7b8cd4d7fb761242e186fc898'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.006381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e626b7b8cd4d7fb761242e186fc898'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.014545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e626b7b8cd4d7fb761242e186fc898'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.015232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e626b7b8cd4d7fb761242e186fc898'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.027786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-131c3d36-75d9-434d-821b-92d534185212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e626b7b8cd4d7fb761242e186fc898'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.040422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1045/2094] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:35 2022] DELETE /identity/v3/users/71e626b7b8cd4d7fb761242e186fc898 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:36.045036 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:36.045398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:36.045908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:36.053264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1050/2095] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:36.063631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:36.065272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.065994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:36.067201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.128942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1046/2096] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:36.131687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c3954-4f99-481e-b249-1087f9d66d12 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:36.138982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:36.140460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.189039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:36.189724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:36.190010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:36.190287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:36.200244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d6ebb-ddcb-4696-a592-7854296ab729 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:36.201328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:36.202878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:36.203966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:36.204676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:36.204955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:36.207413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:36.214352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1051/2097] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:36.215557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:36.217055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:36.226889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1047/2098] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:36.628180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:36.629689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.652625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2006-7511-436f-8a5f-1eb587a4f248 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:36.652812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:36.654319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.658566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:36.659426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:36.659703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:36.660291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] PATH_INFO: `/v3/projects/80a39c0a82e94755904019e1e805cfa2` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:36.661280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a39c0a82e94755904019e1e805cfa2)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:36.663221 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:36.665052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:36.671400 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee82b-c031-4111-bd78-3814206a6437 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:36.678828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:36.680174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:36.680651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:36.681383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] PATH_INFO: `/v3/projects/4e557c2302f5420496db480e0f2953cb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:36.684643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e557c2302f5420496db480e0f2953cb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:36.687500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:36.702017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:36.725328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a39c0a82e94755904019e1e805cfa2', 'request_id': 'req-4cb5d62f-1d00-4c96-842e-4dfdd2436841'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.733014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a39c0a82e94755904019e1e805cfa2', 'request_id': 'req-4cb5d62f-1d00-4c96-842e-4dfdd2436841'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.744593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a39c0a82e94755904019e1e805cfa2', 'request_id': 'req-4cb5d62f-1d00-4c96-842e-4dfdd2436841'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.758189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e557c2302f5420496db480e0f2953cb', 'request_id': 'req-f86eaa42-2e09-4405-ad8a-5546f661e92e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.761912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] The token cache is being invalidate because project 80a39c0a82e94755904019e1e805cfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:36.762258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.763023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.767247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1052/2099] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:36 2022] DELETE /identity/v3/projects/80a39c0a82e94755904019e1e805cfa2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:36.767720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e557c2302f5420496db480e0f2953cb', 'request_id': 'req-f86eaa42-2e09-4405-ad8a-5546f661e92e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.778746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e557c2302f5420496db480e0f2953cb', 'request_id': 'req-f86eaa42-2e09-4405-ad8a-5546f661e92e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.799167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] The token cache is being invalidate because project 4e557c2302f5420496db480e0f2953cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:36.799990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.801055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:36.803058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1048/2100] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:36 2022] DELETE /identity/v3/projects/4e557c2302f5420496db480e0f2953cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:36.807945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:36.809272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:36.902505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d62f-1d00-4c96-842e-4dfdd2436841 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:37.011267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:37.012337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.012636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.012899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] PATH_INFO: `/v3/users/0463d4f8a3274081ae700cd1a3926928` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.014030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=0463d4f8a3274081ae700cd1a3926928)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:37.016047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:37.058010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:37.064927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0463d4f8a3274081ae700cd1a3926928'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.065726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0463d4f8a3274081ae700cd1a3926928'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.066593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0463d4f8a3274081ae700cd1a3926928'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.076665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0463d4f8a3274081ae700cd1a3926928'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.091768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a6274fa4-77a8-4b6a-a690-77f81f3c12ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0463d4f8a3274081ae700cd1a3926928'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.106115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1053/2101] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:36 2022] DELETE /identity/v3/users/0463d4f8a3274081ae700cd1a3926928 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:37.395203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:37.396833 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:37.413583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eaa42-2e09-4405-ad8a-5546f661e92e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:37.515718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:37.516675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.517655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.517655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] PATH_INFO: `/v3/projects/69b3e97931f14afb87f44d9fb6acd121` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.518668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b3e97931f14afb87f44d9fb6acd121)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:37.521074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:37.585349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b3e97931f14afb87f44d9fb6acd121', 'request_id': 'req-f79ed79e-2227-4fc4-96cc-986449d7cebf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.593118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b3e97931f14afb87f44d9fb6acd121', 'request_id': 'req-f79ed79e-2227-4fc4-96cc-986449d7cebf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.601200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b3e97931f14afb87f44d9fb6acd121', 'request_id': 'req-f79ed79e-2227-4fc4-96cc-986449d7cebf'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.619821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] The token cache is being invalidate because project 69b3e97931f14afb87f44d9fb6acd121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:37.620144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.620775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f79ed79e-2227-4fc4-96cc-986449d7cebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:37.622563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1049/2102] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:37 2022] DELETE /identity/v3/projects/69b3e97931f14afb87f44d9fb6acd121 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:37.629918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.630208 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.630486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.654534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:37.655994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:37.717015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1054/2103] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:37.724898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.725190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.725468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.751981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:37.753209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:37.763984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1050/2104] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:37.768427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:37.778453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b239ff-363e-43b8-8f34-6723df95ac96 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:37.783151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:37.783803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.784118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.784393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.785120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:37.785423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:37.787036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:37.798660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1055/2105] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:37.803008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:37.812506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4f747-31c9-4897-8c7b-3af62d98719c None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:37.817656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:37.818299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.818574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.818849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.819535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:37.821775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:37.848270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1051/2106] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:37 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:37.852460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:37.862726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba039f-22c9-4b1c-b975-f55e475ecb64 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:37.935415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:37.936402 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.936687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.936961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.937972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:37.941112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:37.966316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1056/2107] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:37 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:37.970537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:37.976164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:37.983581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f93c5-8764-4f40-8663-956f9a7f1589 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:37.989729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:37.990390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:37.990667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:37.990938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:37.991660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:37.993300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.003978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1052/2108] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:38.008018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.018792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2040ff-42aa-40a3-af63-7e8629878d73 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.024048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957143e-a60c-49f8-a065-73d8377148c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.024937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957143e-a60c-49f8-a065-73d8377148c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.025321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957143e-a60c-49f8-a065-73d8377148c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.025633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957143e-a60c-49f8-a065-73d8377148c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.026507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957143e-a60c-49f8-a065-73d8377148c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.028172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957143e-a60c-49f8-a065-73d8377148c1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.038897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1057/2109] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:38.043414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.053926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cc6d9-a975-46f0-945c-82fdb741b586 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.058722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.059590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.059896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.060157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] PATH_INFO: `/v3/projects/90b0f6bafab64ffd97da7402482cceca/users/2771746701604ab8badbe632b199c1ef/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.075162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b0f6bafab64ffd97da7402482cceca, user_id=2771746701604ab8badbe632b199c1ef, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.077829 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.091917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1053/2110] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:38 2022] PUT /identity/v3/projects/90b0f6bafab64ffd97da7402482cceca/users/2771746701604ab8badbe632b199c1ef/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:38.102138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.102421 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.102694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.130626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] MFA Rules not processed for user `2771746701604ab8badbe632b199c1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:38.131991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:38.142909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:38.201009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1058/2111] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:38.205721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.270706 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cff0a-fc9a-438b-aee8-61a885b391f8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.276309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.276965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.277246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.277493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.278211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.280147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.308643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1054/2112] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:38 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:38.312768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.323925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650177c7-1844-4767-a651-7c7234e62ae2 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.385571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.386235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.386514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.386785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.387487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.389434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.410767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1059/2113] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:38 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:38.416814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.427416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f735b24-f759-45f1-ab6d-3c3d24ea4dda admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.433127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.433776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.434052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.434330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.435048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.436690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.447918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1055/2114] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:38.452641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.469575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726856f-f2b2-4ad2-b5c3-f1b5b2847dda admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.479687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.481135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.481557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.481975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] PATH_INFO: `/v3/projects/687679c8cc1b4d139ab5ba0957a6979a/users/d8a3eb802ab74a318502c31b18634523/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.501096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687679c8cc1b4d139ab5ba0957a6979a, user_id=d8a3eb802ab74a318502c31b18634523, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.503796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.521860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1060/2115] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:38 2022] PUT /identity/v3/projects/687679c8cc1b4d139ab5ba0957a6979a/users/d8a3eb802ab74a318502c31b18634523/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:38.527786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.542955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a976fa9-8698-4673-9ba4-745e0b7e6997 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.608936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc694-3d19-42bd-82f3-dde2ca130b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.609599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc694-3d19-42bd-82f3-dde2ca130b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.609881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc694-3d19-42bd-82f3-dde2ca130b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.610154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc694-3d19-42bd-82f3-dde2ca130b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.610879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc694-3d19-42bd-82f3-dde2ca130b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.612520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc694-3d19-42bd-82f3-dde2ca130b2e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.624023 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1056/2116] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:38.628134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.638883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db72c-b2df-4feb-bbf4-5648e90e1335 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.643684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.644583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.644864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.645140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] PATH_INFO: `/v3/projects/687679c8cc1b4d139ab5ba0957a6979a/users/d8a3eb802ab74a318502c31b18634523/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.647121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=687679c8cc1b4d139ab5ba0957a6979a, user_id=d8a3eb802ab74a318502c31b18634523, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.649784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.660795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1061/2117] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:38 2022] PUT /identity/v3/projects/687679c8cc1b4d139ab5ba0957a6979a/users/d8a3eb802ab74a318502c31b18634523/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:38.672863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.673239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.673574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.704831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] MFA Rules not processed for user `d8a3eb802ab74a318502c31b18634523`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:38.706035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:38.739523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:38.787028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1057/2118] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:38.802029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:38.804086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:38.871825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4f16d-d463-41fc-bdd6-48362bacc78e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:38.875375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0ff6c-7869-4579-a396-ceb1e513e59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:38.876133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0ff6c-7869-4579-a396-ceb1e513e59b service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:38.876408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0ff6c-7869-4579-a396-ceb1e513e59b service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:38.876675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0ff6c-7869-4579-a396-ceb1e513e59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:38.895261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:38.936489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0ff6c-7869-4579-a396-ceb1e513e59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:38.938847 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0ff6c-7869-4579-a396-ceb1e513e59b service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:38.954200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1062/2119] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:45.763131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:45.831132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6700-1a30-4aba-ac2d-db1f5116eae5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:45.835984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:45.836860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:45.837170 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:45.837585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] PATH_INFO: `/v3/users/2771746701604ab8badbe632b199c1ef` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:45.839262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2771746701604ab8badbe632b199c1ef)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:45.841334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:45.890756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2771746701604ab8badbe632b199c1ef'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:45.891413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2771746701604ab8badbe632b199c1ef'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:45.899382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2771746701604ab8badbe632b199c1ef'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:45.900063 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2771746701604ab8badbe632b199c1ef'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:45.914131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2771746701604ab8badbe632b199c1ef'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:45.927590 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1058/2120] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:45 2022] DELETE /identity/v3/users/2771746701604ab8badbe632b199c1ef => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:45.932123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:45.932602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:45.932746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:45.953408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:45.954525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:45.994416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:45.996119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.031924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1063/2121] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:46.041948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:46.044838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.057339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:46.092960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36064-7eae-49c8-8f22-0b6ad6fb19f2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.098303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.099253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.099607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.099978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] PATH_INFO: `/v3/users/4498ef51257044f3aee634437ccb9b0f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.101247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4498ef51257044f3aee634437ccb9b0f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.103434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:46.119882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e15c5d-c9bd-4341-be21-26be0d451418 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.125029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea8c42-e0ee-4be1-9408-458a5e788da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.125850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea8c42-e0ee-4be1-9408-458a5e788da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.126144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea8c42-e0ee-4be1-9408-458a5e788da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.126415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea8c42-e0ee-4be1-9408-458a5e788da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.140041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea8c42-e0ee-4be1-9408-458a5e788da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.141757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea8c42-e0ee-4be1-9408-458a5e788da5 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:46.163344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4498ef51257044f3aee634437ccb9b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.164283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4498ef51257044f3aee634437ccb9b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.164905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1064/2122] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:46.173963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4498ef51257044f3aee634437ccb9b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.175340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4498ef51257044f3aee634437ccb9b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.213584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4498ef51257044f3aee634437ccb9b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.234492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1059/2123] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:45 2022] DELETE /identity/v3/users/4498ef51257044f3aee634437ccb9b0f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:46.239276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.239940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.240315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.260353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:46.261694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.331937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1065/2124] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:46.340707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:46.342221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.425910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b600fd6-d1a7-4d41-96d6-0885b14ef8d8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.429982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.430831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.431334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.432043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.446713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.448597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:46.460492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1060/2125] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:46.560057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:46.561856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.570916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:46.579040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1596eca-27cc-4cc8-8bf8-8afd611ecf8f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.585767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.587469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.588028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.588355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] PATH_INFO: `/v3/projects/90b0f6bafab64ffd97da7402482cceca` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.589883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b0f6bafab64ffd97da7402482cceca)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.592380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:46.666958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b0f6bafab64ffd97da7402482cceca', 'request_id': 'req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.673936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b0f6bafab64ffd97da7402482cceca', 'request_id': 'req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.682713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b0f6bafab64ffd97da7402482cceca', 'request_id': 'req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.698691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] The token cache is being invalidate because project 90b0f6bafab64ffd97da7402482cceca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:46.698975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.699807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.701707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1066/2126] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:46 2022] DELETE /identity/v3/projects/90b0f6bafab64ffd97da7402482cceca => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:46.705690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:46.715967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1af9-cc5b-4cbe-bd7c-10d67a114c3f service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.775674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.776682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.777027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.777389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] PATH_INFO: `/v3/users/d8a3eb802ab74a318502c31b18634523` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.778977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a3eb802ab74a318502c31b18634523)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.781091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:46.786114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:46.788565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:46.790672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.804505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9d2b2-de06-41b2-aa3e-277ae76b32fd admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.809742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.810744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.811031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.811347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] PATH_INFO: `/v3/projects/e223f3f583a443bf993d7112ae8c2d92` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.812589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e223f3f583a443bf993d7112ae8c2d92)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.814527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:46.839300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a3eb802ab74a318502c31b18634523'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.839982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a3eb802ab74a318502c31b18634523'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.849106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a3eb802ab74a318502c31b18634523'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.849693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a3eb802ab74a318502c31b18634523'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.863478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a3eb802ab74a318502c31b18634523'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.867469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e223f3f583a443bf993d7112ae8c2d92', 'request_id': 'req-e8c8f605-98d1-417e-9a01-16cfd2480fc4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.873588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e223f3f583a443bf993d7112ae8c2d92', 'request_id': 'req-e8c8f605-98d1-417e-9a01-16cfd2480fc4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.876295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1061/2127] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:46 2022] DELETE /identity/v3/users/d8a3eb802ab74a318502c31b18634523 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:46.881829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e223f3f583a443bf993d7112ae8c2d92', 'request_id': 'req-e8c8f605-98d1-417e-9a01-16cfd2480fc4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.893202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] The token cache is being invalidate because project e223f3f583a443bf993d7112ae8c2d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:46.893490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.894141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:46.895946 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1067/2128] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:46 2022] DELETE /identity/v3/projects/e223f3f583a443bf993d7112ae8c2d92 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:46.899820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:46.901455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:46.977272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994e343-80d1-4aae-8dc4-fc14355e7c18 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:46.982619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:46.983521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:46.983804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:46.984127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] PATH_INFO: `/v3/users/5025eac87a8e4ec88ec94c4606d65b97` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:46.985383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5025eac87a8e4ec88ec94c4606d65b97)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:46.987335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.040239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5025eac87a8e4ec88ec94c4606d65b97'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.041200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5025eac87a8e4ec88ec94c4606d65b97'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.049802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5025eac87a8e4ec88ec94c4606d65b97'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.050585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5025eac87a8e4ec88ec94c4606d65b97'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.064606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5025eac87a8e4ec88ec94c4606d65b97'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.078410 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1062/2129] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:46 2022] DELETE /identity/v3/users/5025eac87a8e4ec88ec94c4606d65b97 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:47.382971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.385305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:47.463783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.465394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:47.479143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:47.491465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8f605-98d1-417e-9a01-16cfd2480fc4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:47.506600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:47.574348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821edfc-3f9b-490b-a234-217c90ffefb4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:47.579555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:47.580552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.580847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.581288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] PATH_INFO: `/v3/projects/687679c8cc1b4d139ab5ba0957a6979a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.581581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:47.582453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.582738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.582849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687679c8cc1b4d139ab5ba0957a6979a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:47.583009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] PATH_INFO: `/v3/projects/29ee9cc161ab4fb7ab39c606d1520a77` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.584246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ee9cc161ab4fb7ab39c606d1520a77)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:47.586172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.587246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.642396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687679c8cc1b4d139ab5ba0957a6979a', 'request_id': 'req-620cd547-6c06-4f8c-bb5c-90c0b73b4214'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.649340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687679c8cc1b4d139ab5ba0957a6979a', 'request_id': 'req-620cd547-6c06-4f8c-bb5c-90c0b73b4214'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.649464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ee9cc161ab4fb7ab39c606d1520a77', 'request_id': 'req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.656758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687679c8cc1b4d139ab5ba0957a6979a', 'request_id': 'req-620cd547-6c06-4f8c-bb5c-90c0b73b4214'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.660106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ee9cc161ab4fb7ab39c606d1520a77', 'request_id': 'req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.674062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ee9cc161ab4fb7ab39c606d1520a77', 'request_id': 'req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.676113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] The token cache is being invalidate because project 687679c8cc1b4d139ab5ba0957a6979a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:47.676403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.677138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-620cd547-6c06-4f8c-bb5c-90c0b73b4214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.679070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1063/2130] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:47 2022] DELETE /identity/v3/projects/687679c8cc1b4d139ab5ba0957a6979a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:47.684197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705a451-8a22-4865-b336-5e07a32fcfd4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.684482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705a451-8a22-4865-b336-5e07a32fcfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.684753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705a451-8a22-4865-b336-5e07a32fcfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.687390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] The token cache is being invalidate because project 29ee9cc161ab4fb7ab39c606d1520a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:47.687749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.688479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1e9e9d88-3726-4d78-a15a-8991b8d5d8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:47.691950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1068/2131] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:47 2022] DELETE /identity/v3/projects/29ee9cc161ab4fb7ab39c606d1520a77 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:47.695186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12919-7269-4119-be01-2c662bbd505f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.695538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12919-7269-4119-be01-2c662bbd505f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.695902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12919-7269-4119-be01-2c662bbd505f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.705278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9705a451-8a22-4865-b336-5e07a32fcfd4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:47.706500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9705a451-8a22-4865-b336-5e07a32fcfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:47.715583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4dd12919-7269-4119-be01-2c662bbd505f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:47.717222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4dd12919-7269-4119-be01-2c662bbd505f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:47.782927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1064/2132] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:47.786449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1069/2133] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:47.793797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.794046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.794334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.794612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.794736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.795167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.816064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:47.817293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:47.821072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:47.823279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:47.827487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1065/2134] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:47.830988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.835704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1070/2135] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:47.837934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.840725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d38aa-aab8-4581-b258-bd11ff50c9b9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:47.846324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:47.846979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.847545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.847545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.848316 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:47.848626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:47.849772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e7d8-0873-432a-bc55-0c4a71c84b30 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:47.850266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.856526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:47.857492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.858314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.858314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.859404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:47.859931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:47.862434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.864367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1066/2136] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:47.868739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.879956 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1071/2137] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:01:47.884080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.885588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db51e-2bf0-4149-9df4-9844b2b2eb43 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:47.890462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:47.891132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.891571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.891856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.892597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:47.894527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.896229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819f0f3-a04f-4ffe-8837-7242230927bb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:47.902063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:47.903346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:47.903830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:47.904306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:47.905200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:47.905983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:47.907749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:47.929305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1067/2138] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:47.930652 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.943718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1072/2139] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:47.945091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:47.951773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:48.010990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16878b-a745-4e7c-b998-db8b9d840883 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.015724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.016426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.016714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.017378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.017487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb0fd6-b2ab-41f7-a56d-88d689f100a1 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.017682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.019617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.022723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.023483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.023848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.024223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.025553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.027180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.042582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1068/2140] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:48.046194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.055535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1073/2141] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:47 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:48.056799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.061512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bc992-b8a1-4364-8344-943bd9c26a18 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.067012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74eb37-bbaf-4bb5-a936-e49455863a5f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.068273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.068909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.069187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.069507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.070707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.072499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.073138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.074018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.074371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.074753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.075620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.077391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.084906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1069/2142] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:48.088752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1074/2143] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:48.092106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.092531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.102188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90484b16-7d5e-43fa-b060-b92e0f0a2273 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.102522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ac7c-23df-4fa1-9946-a9b18b459935 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.106902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.108095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.108095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.108313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.108964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.109045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.109045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.109186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.109259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.109975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.110673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.111664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.123519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1075/2144] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:48.128600 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1070/2145] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:48.128752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.130303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.138898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17137733-57de-427f-aa8c-ca37540a97d9 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.143327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20582ed5-a2d0-4c4d-9c50-c859316bb342 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.144878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae7b0e-1e8c-4cfb-a7d7-0735e0894ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.145874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae7b0e-1e8c-4cfb-a7d7-0735e0894ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.146161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae7b0e-1e8c-4cfb-a7d7-0735e0894ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.146614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae7b0e-1e8c-4cfb-a7d7-0735e0894ac0 admin admin] PATH_INFO: `/v3/projects/73354670c36547d087a90afc72b0a9e9/users/a58f585fc2f14c118880d3015cfec0e3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.148331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53f8cf-79c9-45fb-a064-e9eecebcab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.149358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f8cf-79c9-45fb-a064-e9eecebcab9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.149811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f8cf-79c9-45fb-a064-e9eecebcab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.150087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f8cf-79c9-45fb-a064-e9eecebcab9e admin admin] PATH_INFO: `/v3/projects/46ae600a17f9437d9e62475696c106ca/users/64fe0b9043ad479284163d04a90b842f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.164951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae7b0e-1e8c-4cfb-a7d7-0735e0894ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73354670c36547d087a90afc72b0a9e9, user_id=a58f585fc2f14c118880d3015cfec0e3, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.166884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53f8cf-79c9-45fb-a064-e9eecebcab9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ae600a17f9437d9e62475696c106ca, user_id=64fe0b9043ad479284163d04a90b842f, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.167996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae7b0e-1e8c-4cfb-a7d7-0735e0894ac0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.169703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53f8cf-79c9-45fb-a064-e9eecebcab9e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.185012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1071/2146] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:48 2022] PUT /identity/v3/projects/46ae600a17f9437d9e62475696c106ca/users/64fe0b9043ad479284163d04a90b842f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:48.185232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1076/2147] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:48 2022] PUT /identity/v3/projects/73354670c36547d087a90afc72b0a9e9/users/a58f585fc2f14c118880d3015cfec0e3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:48.189657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.189998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.190279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.199677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.200054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.200323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.215622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:48.216940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:48.228282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] MFA Rules not processed for user `a58f585fc2f14c118880d3015cfec0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:48.229661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:48.296595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1072/2148] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:48.307264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1077/2149] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:48.307918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.308069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.310105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:48.318771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ef6a7-1ae1-44a0-b257-15b229cf96ff None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.324979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.325692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.325971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.326479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.327400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.329358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.356253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:48.357606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1078/2150] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:48.361627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.382194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced3a1-0677-4fdd-902f-3bc81a16036b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.382968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:48.434147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213d116-976c-49c3-be9d-4a545fce3719 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.439817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.440922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.441259 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.441640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.442377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.445894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.449707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.450544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.450895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.451136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.464625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.466092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.473453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1079/2151] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:48.477681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.482225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1073/2152] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:48.488848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d1426-c00e-4369-a65e-2346c090540b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.499176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.500520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.500625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.500904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.502419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.505633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.519695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1080/2153] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:48.523764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.534589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274149a2-6d88-417e-adca-3f61f1952286 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.540229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665e02b-0b5c-4775-98cd-1d285dc964e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.541323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665e02b-0b5c-4775-98cd-1d285dc964e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.541452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665e02b-0b5c-4775-98cd-1d285dc964e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.541748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665e02b-0b5c-4775-98cd-1d285dc964e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.542547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665e02b-0b5c-4775-98cd-1d285dc964e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.544251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665e02b-0b5c-4775-98cd-1d285dc964e0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.557852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1074/2154] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:48.561797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.573517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899cf69-5072-4904-9d57-4e0c3a19c83c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.578350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.579472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.579689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.579958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] PATH_INFO: `/v3/projects/57bdc4f508dc4af1916c7b9d327ea1de/users/2aa3c1fca03745b7829505384b45db10/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.611353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bdc4f508dc4af1916c7b9d327ea1de, user_id=2aa3c1fca03745b7829505384b45db10, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.617511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.639748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1081/2155] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:48 2022] PUT /identity/v3/projects/57bdc4f508dc4af1916c7b9d327ea1de/users/2aa3c1fca03745b7829505384b45db10/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:48.652392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.653815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.654168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.699623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] MFA Rules not processed for user `2aa3c1fca03745b7829505384b45db10`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:48.701339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:48.820940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1075/2156] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:48.826709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.909088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8bbdb-12d7-4d5e-b3a2-ac8a7f28043a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:48.916238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:48.917015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:48.917366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:48.917711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:48.918607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:48.921050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:48.951678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1082/2157] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:48.956165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:48.969516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdfdbe-1cf6-4072-8852-0abc94046b70 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.044558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.045252 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.045587 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.045865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.047431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.050472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.087442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:49.097223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1076/2158] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:48 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:49.101779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852701f-2054-499c-9ec4-24a92974f172 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.107346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.108068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.108570 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.108681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.109330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.111131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.123975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1083/2159] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:49.128391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:49.135921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:49.145813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c289c05-bf89-4e12-b7b2-d94dfe7851a5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.152699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.154118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.154118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.154376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] PATH_INFO: `/v3/projects/8d1d92371bc74eddb5ba6db7b46bfc10/users/021be7076965420fa22de5afa90aa5e6/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.176818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1d92371bc74eddb5ba6db7b46bfc10, user_id=021be7076965420fa22de5afa90aa5e6, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.179698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.201976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1077/2160] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:49 2022] PUT /identity/v3/projects/8d1d92371bc74eddb5ba6db7b46bfc10/users/021be7076965420fa22de5afa90aa5e6/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:49.203597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:49.213775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421902a-e1ae-4a42-89d1-ea025dab97d4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.262297 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:49.286928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc99fa-8586-4bdb-9c82-9ea86fade8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.295590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc99fa-8586-4bdb-9c82-9ea86fade8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.295917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc99fa-8586-4bdb-9c82-9ea86fade8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.296201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc99fa-8586-4bdb-9c82-9ea86fade8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.296969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc99fa-8586-4bdb-9c82-9ea86fade8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.298720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc99fa-8586-4bdb-9c82-9ea86fade8e6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.326371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1084/2161] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:49.331224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:49.346832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784669f-53f6-4616-b816-f669f362f7a5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.356080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e092b5-68ce-4e14-939e-46630279ae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.357698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e092b5-68ce-4e14-939e-46630279ae2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.357987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e092b5-68ce-4e14-939e-46630279ae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.358259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e092b5-68ce-4e14-939e-46630279ae2a admin admin] PATH_INFO: `/v3/projects/8d1d92371bc74eddb5ba6db7b46bfc10/users/021be7076965420fa22de5afa90aa5e6/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.360354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e092b5-68ce-4e14-939e-46630279ae2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1d92371bc74eddb5ba6db7b46bfc10, user_id=021be7076965420fa22de5afa90aa5e6, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.363198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e092b5-68ce-4e14-939e-46630279ae2a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.378446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1078/2162] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:49 2022] PUT /identity/v3/projects/8d1d92371bc74eddb5ba6db7b46bfc10/users/021be7076965420fa22de5afa90aa5e6/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:49.389535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.389841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.390158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.404486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.404895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.405536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.421270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] MFA Rules not processed for user `021be7076965420fa22de5afa90aa5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:49.422479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:49.452077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] MFA Rules not processed for user `64fe0b9043ad479284163d04a90b842f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:49.452823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:49.529629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1085/2163] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:49.541690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:49.543145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:49.550929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1079/2164] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:49.563296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:49.564809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:49.645427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8248bc-e9fe-44a3-b1fe-097a64c6f307 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.651745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d2323-84cf-4200-a9a4-da72b53d4040 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:49.654312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545c4a8-6c9f-460d-b268-ea9d4995e942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.654993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545c4a8-6c9f-460d-b268-ea9d4995e942 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.655530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545c4a8-6c9f-460d-b268-ea9d4995e942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.655810 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545c4a8-6c9f-460d-b268-ea9d4995e942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.661396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf5a33-a168-4225-8b13-0ac355afd875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:49.662198 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5a33-a168-4225-8b13-0ac355afd875 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:49.662377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5a33-a168-4225-8b13-0ac355afd875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:49.663659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5a33-a168-4225-8b13-0ac355afd875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:49.676277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545c4a8-6c9f-460d-b268-ea9d4995e942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.677751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545c4a8-6c9f-460d-b268-ea9d4995e942 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.704980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1080/2165] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:49.774360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf5a33-a168-4225-8b13-0ac355afd875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:49.778887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf5a33-a168-4225-8b13-0ac355afd875 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:49.793729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1086/2166] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:51.320098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ee21-a6aa-4d62-89ba-6ea82754762f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:51.320530 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ee21-a6aa-4d62-89ba-6ea82754762f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:51.320636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ee21-a6aa-4d62-89ba-6ea82754762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:51.349292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f452ee21-a6aa-4d62-89ba-6ea82754762f None None] MFA Rules not processed for user `64fe0b9043ad479284163d04a90b842f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:51.350534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f452ee21-a6aa-4d62-89ba-6ea82754762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:51.363206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1081/2167] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:51.493325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374830aa-a698-479c-92a8-af54ee5276cc None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:51.493823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374830aa-a698-479c-92a8-af54ee5276cc None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:51.493919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374830aa-a698-479c-92a8-af54ee5276cc None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:51.496027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1087/2168] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:01:51.512665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:51.513219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:51.513796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:51.533857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:51.535268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:51.574927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374830aa-a698-479c-92a8-af54ee5276cc None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:51.590117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374830aa-a698-479c-92a8-af54ee5276cc None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:51.593696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:51.594449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:51.594800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:51.595102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:51.608387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:51.627858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:51.657445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1082/2169] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:01:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:51.663101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:51.673683 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:51.708629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:51.710071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:51.722541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1088/2170] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:51.724468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:51.726054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:51.752863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe7219-4c7b-4772-bc06-3158f3cc8b40 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:51.758530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:51.760588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:51.760871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:51.761145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:51.778501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:51.780299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:51.788263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25efef6-c090-4b11-a13a-add72f9a8dfc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:51.793722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:51.794804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:51.795259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:51.795678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] PATH_INFO: `/v3/users/b7a8708943a94915970a0ff89b9f17e3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:51.798209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a8708943a94915970a0ff89b9f17e3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:51.800787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:51.816296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1089/2171] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:51.874655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a8708943a94915970a0ff89b9f17e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:51.875401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a8708943a94915970a0ff89b9f17e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:51.886541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a8708943a94915970a0ff89b9f17e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:51.887222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a8708943a94915970a0ff89b9f17e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:51.917388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-89c115c8-0469-44bb-af19-9ed5288f2419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a8708943a94915970a0ff89b9f17e3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:51.943536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1083/2172] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:51 2022] DELETE /identity/v3/users/b7a8708943a94915970a0ff89b9f17e3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:52.264052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.276802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3dd9d-82c0-4a1e-889e-56deb9439497 service placement] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.375446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.376571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.376908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.377218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] PATH_INFO: `/v3/projects/c06b1cb0fcef4cbba01134acdba481be` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.378807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06b1cb0fcef4cbba01134acdba481be)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.381121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.440565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06b1cb0fcef4cbba01134acdba481be', 'request_id': 'req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:52.448585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06b1cb0fcef4cbba01134acdba481be', 'request_id': 'req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:52.457678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06b1cb0fcef4cbba01134acdba481be', 'request_id': 'req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:52.470012 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] The token cache is being invalidate because project c06b1cb0fcef4cbba01134acdba481be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:52.470155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:52.470922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:52.472920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1090/2173] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:52 2022] DELETE /identity/v3/projects/c06b1cb0fcef4cbba01134acdba481be => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:52.479051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aeb8c-9c59-4cf5-8ff2-f0be3141ea4f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.479547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aeb8c-9c59-4cf5-8ff2-f0be3141ea4f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.479846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aeb8c-9c59-4cf5-8ff2-f0be3141ea4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.509929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-670aeb8c-9c59-4cf5-8ff2-f0be3141ea4f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:52.511210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-670aeb8c-9c59-4cf5-8ff2-f0be3141ea4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:52.547433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.549399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:52.586701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1084/2174] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:52.595317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.596048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.596158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:52.596258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.610179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:52.616666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:52.616853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbf45e-1d9c-4ace-8018-c2660bf90e2d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.618948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:52.623125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.623918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.624270 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.624619 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.626816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:52.632456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1085/2175] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:52.637420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.655581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821820c-c5b4-422c-8b40-97fff34d4ff9 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.660708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.661876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.661876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.662080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.662818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:52.663158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.665477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.686930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1086/2176] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:52.690659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.710988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b9cff-a387-4a3a-9f18-a5467075e515 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.718916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.719811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.720777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.721079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.722193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.722896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.724288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.724409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.741058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1091/2177] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:52.763342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1087/2178] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:52 2022] POST /identity/v3/projects => generated 361 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:52.764396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.775968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb45ceb-8ef6-491c-9337-661963dd8367 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.877118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.878082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.878716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.878716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.879458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.881480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.903909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1092/2179] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:52 2022] POST /identity/v3/users => generated 281 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:52.907838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.918161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd28a-9de6-4fe8-b538-b7a3eb3e4fbb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.923382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.924344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.924691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.925099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.926104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.928476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.939393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1088/2180] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:52.944940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.956457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba85c-ad1d-4881-bd14-583832ce9322 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.962165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301733-490c-4705-ac11-6192f5f8aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.962948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301733-490c-4705-ac11-6192f5f8aa85 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.963231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301733-490c-4705-ac11-6192f5f8aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:52.963504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301733-490c-4705-ac11-6192f5f8aa85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:52.964336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301733-490c-4705-ac11-6192f5f8aa85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:52.965955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301733-490c-4705-ac11-6192f5f8aa85 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:52.976922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1093/2181] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:52.981337 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:52.992832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb5576-48d8-40af-a2c8-ca9f816f51ea admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:52.999157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:52.999658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:52.999965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.000239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] PATH_INFO: `/v3/projects/f42b57bd63844e6b83abefca03f0b9ec/users/47634944d2444db69d32881902f0c2bb/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.017999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42b57bd63844e6b83abefca03f0b9ec, user_id=47634944d2444db69d32881902f0c2bb, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.020935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.035958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1089/2182] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:52 2022] PUT /identity/v3/projects/f42b57bd63844e6b83abefca03f0b9ec/users/47634944d2444db69d32881902f0c2bb/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:53.046784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.047491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.047773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.078192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] MFA Rules not processed for user `47634944d2444db69d32881902f0c2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:53.080033 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:53.151258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:53.164881 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1094/2183] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:53.169231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.239406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e371f-8173-4a05-b9fb-06c8fcefe6c9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.245914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.246664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.247010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.247346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.248139 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.250123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.302633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1090/2184] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:53 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:53.303572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.324227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722199-77ed-44eb-8312-62ce636488a0 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.406804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.407601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.407903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.408181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.408891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.410862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.433371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1095/2185] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:53 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:53.437440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.451365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4665bff-4ea0-4494-82a0-1b11e3488344 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.456499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.457827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.458281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.458739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.459724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.461466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.474152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1091/2186] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:53.478410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.489027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951671-0839-42c5-9003-f56a294eee0b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.494311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.495234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.495515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.495788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] PATH_INFO: `/v3/projects/75bd08c6486e46d7800f377bd78aa5a2/users/b901503025fa479193c20a64ba400b64/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.511903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75bd08c6486e46d7800f377bd78aa5a2, user_id=b901503025fa479193c20a64ba400b64, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.514566 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.528813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1096/2187] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:53 2022] PUT /identity/v3/projects/75bd08c6486e46d7800f377bd78aa5a2/users/b901503025fa479193c20a64ba400b64/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:53.532852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.543287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329829a4-1121-4bcd-b9d6-e6df99fce9cc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.616028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec1ddd-7946-42e6-9d04-8f152bc8af7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.616703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec1ddd-7946-42e6-9d04-8f152bc8af7d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.616981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec1ddd-7946-42e6-9d04-8f152bc8af7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.617254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec1ddd-7946-42e6-9d04-8f152bc8af7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.617984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec1ddd-7946-42e6-9d04-8f152bc8af7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.619658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec1ddd-7946-42e6-9d04-8f152bc8af7d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.624995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:53.632386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1092/2188] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:53.635744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.646366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa45ca8-c5d0-4d18-81a4-ef82bdd7ab04 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.651446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.652466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.652742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.653018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] PATH_INFO: `/v3/projects/75bd08c6486e46d7800f377bd78aa5a2/users/b901503025fa479193c20a64ba400b64/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.655483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75bd08c6486e46d7800f377bd78aa5a2, user_id=b901503025fa479193c20a64ba400b64, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.658192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.670540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1097/2189] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:53 2022] PUT /identity/v3/projects/75bd08c6486e46d7800f377bd78aa5a2/users/b901503025fa479193c20a64ba400b64/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:53.680630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.680932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.681210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.708956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] MFA Rules not processed for user `b901503025fa479193c20a64ba400b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:53.710395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:53.783887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1093/2190] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:53.796360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:53.798550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:53.861308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93aabc-95ef-4fcb-b2a0-de3cedbd9c98 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:53.865318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:53.865943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:53.866240 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:53.866489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:53.877137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:53.878580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:53.890598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1098/2191] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:54.411713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:54.413473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:54.429316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:54.485742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:54.502184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa8aad-44bb-45b4-9097-058f31034892 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:54.505500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781f85c-16e8-4303-8628-9064a788c157 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:54.508187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:54.509098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:54.509377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:54.509655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] PATH_INFO: `/v3/users/47634944d2444db69d32881902f0c2bb` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:54.510433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:54.510903 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47634944d2444db69d32881902f0c2bb)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:54.511641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:54.512034 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:54.512416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] PATH_INFO: `/v3/users/66a9dfdc845c4a14b0f4e4b536722360` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:54.512891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:54.513792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a9dfdc845c4a14b0f4e4b536722360)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:54.515796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:54.566345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47634944d2444db69d32881902f0c2bb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.567258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47634944d2444db69d32881902f0c2bb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.567859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47634944d2444db69d32881902f0c2bb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.575487 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a9dfdc845c4a14b0f4e4b536722360'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.576869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a9dfdc845c4a14b0f4e4b536722360'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.580407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47634944d2444db69d32881902f0c2bb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.587860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a9dfdc845c4a14b0f4e4b536722360'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.588594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a9dfdc845c4a14b0f4e4b536722360'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.594952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ba39e2ce-22e7-4ecd-bcaf-05a2b4ef68a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47634944d2444db69d32881902f0c2bb'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.601408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3108459f-3228-4f77-90e1-52ff6ce40adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a9dfdc845c4a14b0f4e4b536722360'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:54.615117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1094/2192] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:54 2022] DELETE /identity/v3/users/66a9dfdc845c4a14b0f4e4b536722360 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:54.616386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1099/2193] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:54 2022] DELETE /identity/v3/users/47634944d2444db69d32881902f0c2bb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:54.619625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:54.619935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:54.620448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:54.620957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:54.621306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:54.621648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:54.627070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:54.638845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:54.639909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:54.640414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:54.641284 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:54.713289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1100/2194] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:54.721155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1095/2195] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:54.725097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:54.726548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:54.728187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:54.729669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:54.793885 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cafc31-425d-4d7a-97e9-527e70960d64 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:54.794095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36102-c4c8-413e-bd3f-8cfd29b6773f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:54.798473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:54.798607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:54.799134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:54.799360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:54.799461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:54.800049 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:54.800159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:54.800245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:54.810876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:54.812252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:54.812408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:54.813797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:54.823155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1096/2196] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:54.825559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1101/2197] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:55.312372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.314181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:55.332022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c95a5-183a-47ef-92e6-7ed8fbcb5d28 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.343727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.345158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:55.351482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.352621 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.352905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.353308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] PATH_INFO: `/v3/projects/f42b57bd63844e6b83abefca03f0b9ec` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.354945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42b57bd63844e6b83abefca03f0b9ec)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.361438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.362209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801b175-9c0c-472d-90f0-9461777c450d service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.368151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.369803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.370924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.371439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] PATH_INFO: `/v3/projects/414f8213431344ce93e298181f5572b6` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.383656 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] RBAC: Authorizing `identity:delete_project(project_id=414f8213431344ce93e298181f5572b6)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.386141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.397906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:55.432193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42b57bd63844e6b83abefca03f0b9ec', 'request_id': 'req-4d366810-c415-4327-b836-094811fcc803'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.452270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42b57bd63844e6b83abefca03f0b9ec', 'request_id': 'req-4d366810-c415-4327-b836-094811fcc803'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.459767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42b57bd63844e6b83abefca03f0b9ec', 'request_id': 'req-4d366810-c415-4327-b836-094811fcc803'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.478675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] The token cache is being invalidate because project f42b57bd63844e6b83abefca03f0b9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:55.479042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.480144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.481773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1097/2198] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:55 2022] DELETE /identity/v3/projects/f42b57bd63844e6b83abefca03f0b9ec => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:55.485946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.486760 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414f8213431344ce93e298181f5572b6', 'request_id': 'req-0abdcbee-796a-48e1-8001-ae1a93a8776e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.488634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:55.500182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414f8213431344ce93e298181f5572b6', 'request_id': 'req-0abdcbee-796a-48e1-8001-ae1a93a8776e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.510070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414f8213431344ce93e298181f5572b6', 'request_id': 'req-0abdcbee-796a-48e1-8001-ae1a93a8776e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.522695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] The token cache is being invalidate because project 414f8213431344ce93e298181f5572b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:55.523090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.523905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0abdcbee-796a-48e1-8001-ae1a93a8776e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.526298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1102/2199] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:55 2022] DELETE /identity/v3/projects/414f8213431344ce93e298181f5572b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:55.530843 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd311f-563f-4212-8365-b54fdfc66498 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.531127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd311f-563f-4212-8365-b54fdfc66498 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.531400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd311f-563f-4212-8365-b54fdfc66498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.545270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:55.553215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e3cd311f-563f-4212-8365-b54fdfc66498 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:55.554723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e3cd311f-563f-4212-8365-b54fdfc66498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:55.603660 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d366810-c415-4327-b836-094811fcc803 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.623941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1103/2200] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:55.629990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.630281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.630563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.651193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:55.652630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:55.662301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1104/2201] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:55.666603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.676699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a0406-f391-4160-9b03-7b57eaad3532 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.677971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.679452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.680196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.680648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] PATH_INFO: `/v3/users/b901503025fa479193c20a64ba400b64` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.682567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b901503025fa479193c20a64ba400b64)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.682716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.683485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.683895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.684214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.684774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.685101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:55.685507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.687443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.705334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1105/2202] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:55.710253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.725479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175689a-8020-462a-8508-1ebe613172c5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.731375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.732260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.732726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.733106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.748709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.748709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.763896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b901503025fa479193c20a64ba400b64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.764699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b901503025fa479193c20a64ba400b64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.774096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b901503025fa479193c20a64ba400b64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.775019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b901503025fa479193c20a64ba400b64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.777764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1106/2203] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:55 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:55.781454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.783187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:55.795247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b901503025fa479193c20a64ba400b64'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:55.809341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1098/2204] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:55 2022] DELETE /identity/v3/users/b901503025fa479193c20a64ba400b64 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:55.862567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:55.877030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1ff87-d35b-49ce-811b-2121f2b48d88 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.938096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.938775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.939067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.939333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.940075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.942092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.966981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1107/2205] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:55 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:55.968502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:55.979233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b3d9-9ba6-43ff-988f-adac04a82bb9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:55.984476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:55.985202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:55.985489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:55.985765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:55.986540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:55.988161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:55.999162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1099/2206] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:56.003214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.013396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675120a-b7bb-4ba8-b963-c5ae5059dbfd admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.018071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ec2f4-b966-4d75-8a70-d07f5ed7bed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.018733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ec2f4-b966-4d75-8a70-d07f5ed7bed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.019011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ec2f4-b966-4d75-8a70-d07f5ed7bed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.019281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ec2f4-b966-4d75-8a70-d07f5ed7bed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.020047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ec2f4-b966-4d75-8a70-d07f5ed7bed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.021643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ec2f4-b966-4d75-8a70-d07f5ed7bed3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.033023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1108/2207] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:56.036998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.046994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12208f5-6e8c-4946-b46c-9ebd3173b570 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.052025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.052906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.053169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.053558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] PATH_INFO: `/v3/projects/2070ae6daaac45af9a3dda314428077f/users/b161bf44482c4687ae987272dcd59496/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.068968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2070ae6daaac45af9a3dda314428077f, user_id=b161bf44482c4687ae987272dcd59496, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.072008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.087920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1100/2208] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:56 2022] PUT /identity/v3/projects/2070ae6daaac45af9a3dda314428077f/users/b161bf44482c4687ae987272dcd59496/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:56.099808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.100067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.100403 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.120269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.123629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.131190 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] MFA Rules not processed for user `b161bf44482c4687ae987272dcd59496`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:56.131901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.160605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:56.202147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3de22-46ba-4e4f-be42-02087d9f3554 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.207053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.207983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.208248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.208538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] PATH_INFO: `/v3/projects/75bd08c6486e46d7800f377bd78aa5a2` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.209721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75bd08c6486e46d7800f377bd78aa5a2)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.211623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.215774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1109/2209] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:56.220966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.231764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b5491-f0cf-4da2-a592-7124bbf14aaf None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.236329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.236982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.237260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.237536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.238264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.240223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.262586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75bd08c6486e46d7800f377bd78aa5a2', 'request_id': 'req-7faa5b73-aa43-45a0-96c4-d828f642aec2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:56.267677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1110/2210] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:56.268376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75bd08c6486e46d7800f377bd78aa5a2', 'request_id': 'req-7faa5b73-aa43-45a0-96c4-d828f642aec2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:56.272176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.280396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75bd08c6486e46d7800f377bd78aa5a2', 'request_id': 'req-7faa5b73-aa43-45a0-96c4-d828f642aec2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:56.302841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] The token cache is being invalidate because project 75bd08c6486e46d7800f377bd78aa5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:56.303150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:56.304161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-7faa5b73-aa43-45a0-96c4-d828f642aec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:56.305790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1101/2211] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:56 2022] DELETE /identity/v3/projects/75bd08c6486e46d7800f377bd78aa5a2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:56.310584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29710051-0ffe-41e3-8ec2-042551278905 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.310873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29710051-0ffe-41e3-8ec2-042551278905 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.311142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29710051-0ffe-41e3-8ec2-042551278905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.325083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:56.331754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-29710051-0ffe-41e3-8ec2-042551278905 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:56.332981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-29710051-0ffe-41e3-8ec2-042551278905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.340937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca6aa3-35c0-49b2-8363-1f3f1a1bd9c8 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.341714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1102/2212] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:56.345565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.346263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.346539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.346808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.347501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.349462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.350698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.350979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.351250 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.369836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:56.371013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.371821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1111/2213] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:56.375962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.377703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.381370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1103/2214] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:56.385453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.390858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e11ea-9c2a-4af2-bc04-27a1d0ca299a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.394362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a980a0-bd5c-4f8f-b047-cca3434f6d28 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.396003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.396671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.396952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.397230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.397957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.398976 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.400028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.400153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.400566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.401004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.401742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:56.402080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.403772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.411165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1112/2215] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:56.415349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.418074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1104/2216] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:01:56.421441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.425568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2249-e052-4589-98fe-7edb51779f48 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.430344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.431010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.431293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.431570 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.431982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de9fdb-1171-4901-af8c-77207152c5ab admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.432332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.433992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.436725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.437361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.437641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.437917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.438605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.440529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.445523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1113/2217] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:56.450200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.461164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefde05-dc6e-481c-a6f0-ec4bbeb4f439 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.467552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7caa2-5943-4d60-917c-6d04cc00bad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.468498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7caa2-5943-4d60-917c-6d04cc00bad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.468776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7caa2-5943-4d60-917c-6d04cc00bad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.469050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7caa2-5943-4d60-917c-6d04cc00bad5 admin admin] PATH_INFO: `/v3/projects/9008f7b0e7a04481b257d4f8920e7919/users/b4aed351fa9145bfbc7403273cab561e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.471978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1105/2218] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:56.476160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.491199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7caa2-5943-4d60-917c-6d04cc00bad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9008f7b0e7a04481b257d4f8920e7919, user_id=b4aed351fa9145bfbc7403273cab561e, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.493887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7caa2-5943-4d60-917c-6d04cc00bad5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.509540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1114/2219] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:56 2022] PUT /identity/v3/projects/9008f7b0e7a04481b257d4f8920e7919/users/b4aed351fa9145bfbc7403273cab561e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:56.519660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.519989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.520295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.546808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc5067-44b7-4282-ae31-752544ec0e5b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.550578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] MFA Rules not processed for user `b4aed351fa9145bfbc7403273cab561e`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:56.551910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.554942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:56.626168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.627122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.627408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.627797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.628475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.630405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.630790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1115/2220] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:56.635539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.648015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55d24a-67af-4e70-b0dc-d47d17b79139 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.653773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.654551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.654906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.655242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.656050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.658053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.658643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1106/2221] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:56.662987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.674374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f09f3d-8bff-4a43-83c5-0763d711a220 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.679801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.680715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.681031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.681358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.682199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.683836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.689003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1116/2222] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:56.693068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.699269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1107/2223] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:56.700278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.710562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1acf-e479-4dac-b2ef-700c0166e5f4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.757279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:56.761788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cd41e-2852-4db3-b543-17696a128b8b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.770241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.771178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.771677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.772130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.772994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.775350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.787811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.788545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.788827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.789099 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.789887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.791640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.800588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1117/2224] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:56.805481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.806281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1108/2225] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:56.807194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.818020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae773bfe-ea63-479b-9bb6-ad78c18ed135 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.818687 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e1024-cf20-4f3c-a6fc-357f7b5c1a13 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.824541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.824739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.825535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.825820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.826100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] PATH_INFO: `/v3/projects/31468a1725ed43f6b93208484155d1c6/users/25d5cecef695490ebf4319f282496220/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.826494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.826926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.827338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.828252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.833213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.844945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1118/2226] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:56.846296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.849547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] RBAC: Authorizing `identity:create_grant(project_id=31468a1725ed43f6b93208484155d1c6, user_id=25d5cecef695490ebf4319f282496220, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.849698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:56.852619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.863828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a28d-c572-4acd-a3cc-d75a167793de admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.868488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1109/2227] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:56 2022] PUT /identity/v3/projects/31468a1725ed43f6b93208484155d1c6/users/25d5cecef695490ebf4319f282496220/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:56.878845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:56.941471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.942293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.942577 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.942849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] PATH_INFO: `/v3/users/a58f585fc2f14c118880d3015cfec0e3` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.944294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58f585fc2f14c118880d3015cfec0e3)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.946786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.953084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21fd0b-074f-499e-975c-2c8f6d44636e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:56.957788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed096d-400d-4049-9803-f98103e89a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:56.958676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed096d-400d-4049-9803-f98103e89a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.958953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed096d-400d-4049-9803-f98103e89a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.959218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed096d-400d-4049-9803-f98103e89a30 admin admin] PATH_INFO: `/v3/projects/9686d7de78ce4b1abd29547aedbc4605/users/8536151b153d4223992e08de1ad752d3/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:56.962642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:56.976145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed096d-400d-4049-9803-f98103e89a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9686d7de78ce4b1abd29547aedbc4605, user_id=8536151b153d4223992e08de1ad752d3, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:56.979035 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed096d-400d-4049-9803-f98103e89a30 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:56.997265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1110/2228] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:56 2022] PUT /identity/v3/projects/9686d7de78ce4b1abd29547aedbc4605/users/8536151b153d4223992e08de1ad752d3/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:56.997265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0910b05-ed09-4d56-b89f-2a3ecc6cb969 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:56.997265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0910b05-ed09-4d56-b89f-2a3ecc6cb969 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:56.997265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0910b05-ed09-4d56-b89f-2a3ecc6cb969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.015332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58f585fc2f14c118880d3015cfec0e3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.016064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58f585fc2f14c118880d3015cfec0e3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.016904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58f585fc2f14c118880d3015cfec0e3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.026707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58f585fc2f14c118880d3015cfec0e3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.029015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b0910b05-ed09-4d56-b89f-2a3ecc6cb969 None None] MFA Rules not processed for user `25d5cecef695490ebf4319f282496220`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:57.030330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b0910b05-ed09-4d56-b89f-2a3ecc6cb969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.042425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58f585fc2f14c118880d3015cfec0e3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.059575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1119/2229] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:56 2022] DELETE /identity/v3/users/a58f585fc2f14c118880d3015cfec0e3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:57.061653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.063653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.103000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1111/2230] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:57.104626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.105047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.105470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.123299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:57.124596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.143159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:57.149482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6204d63-c228-4945-8524-36308486f868 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:57.161853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:57.162826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.163110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.163384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.164148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:57.165868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:57.180020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1120/2231] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:57.180020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.181340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.205642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1112/2232] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:57.209722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.221405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa2735-4c98-4a44-b71d-17fe68263559 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:57.228620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:57.229909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.229909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.230266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] PATH_INFO: `/v3/projects/9686d7de78ce4b1abd29547aedbc4605/users/8536151b153d4223992e08de1ad752d3/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.232425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9686d7de78ce4b1abd29547aedbc4605, user_id=8536151b153d4223992e08de1ad752d3, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:57.235212 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:57.242651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965e8e-7222-43ed-b7ca-04df0dd47874 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:57.246941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1113/2233] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:57 2022] PUT /identity/v3/projects/9686d7de78ce4b1abd29547aedbc4605/users/8536151b153d4223992e08de1ad752d3/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:57.248028 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.250585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.304291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c8021-c766-45dd-a8d7-7dfc5b117f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:57.304953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8021-c766-45dd-a8d7-7dfc5b117f71 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.305401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8021-c766-45dd-a8d7-7dfc5b117f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.305672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8021-c766-45dd-a8d7-7dfc5b117f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.315513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafdddc-a254-49a1-8f0c-678d4bf3a911 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:57.319101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:57.320094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.320373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.320645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.335958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:57.371942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c8021-c766-45dd-a8d7-7dfc5b117f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:57.373897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c8021-c766-45dd-a8d7-7dfc5b117f71 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:57.384719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1121/2234] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:57.386790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.387321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.387743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.391093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:57.392774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:57.403535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1114/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:57.423289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] MFA Rules not processed for user `8536151b153d4223992e08de1ad752d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:57.424841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.487078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:57.520172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1122/2236] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:57.530064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.532137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.548678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10fde1-318a-4d0d-8e5d-a37f84986f4b service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:57.554157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:57.554855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.555100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.555501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.560918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.639476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:57.641075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:57.657225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1115/2237] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:01:57.779292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.784123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:57.801155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132e313-ba7d-4cbf-9d55-915fd0bf81ca None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:57.807311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:57.808418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:57.808794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:57.809191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] PATH_INFO: `/v3/projects/73354670c36547d087a90afc72b0a9e9` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:57.810641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73354670c36547d087a90afc72b0a9e9)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:57.813031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:57.885301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73354670c36547d087a90afc72b0a9e9', 'request_id': 'req-066c3ed6-f68f-4655-98df-7063266a5383'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.892019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73354670c36547d087a90afc72b0a9e9', 'request_id': 'req-066c3ed6-f68f-4655-98df-7063266a5383'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.901940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73354670c36547d087a90afc72b0a9e9', 'request_id': 'req-066c3ed6-f68f-4655-98df-7063266a5383'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.924817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] The token cache is being invalidate because project 73354670c36547d087a90afc72b0a9e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:57.925234 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.925937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:57.927944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1123/2238] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:57 2022] DELETE /identity/v3/projects/73354670c36547d087a90afc72b0a9e9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:57.932987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:57.946440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6c188-628a-4bfb-9f7e-9a4044443d89 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:58.015140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:58.016219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:58.016567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:58.016908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] PATH_INFO: `/v3/users/2aa3c1fca03745b7829505384b45db10` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:58.018249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa3c1fca03745b7829505384b45db10)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:58.020365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:58.104998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa3c1fca03745b7829505384b45db10'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.106914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa3c1fca03745b7829505384b45db10'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.109514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:58.114775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa3c1fca03745b7829505384b45db10'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.115543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa3c1fca03745b7829505384b45db10'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.137871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa3c1fca03745b7829505384b45db10'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.150534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1116/2239] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:57 2022] DELETE /identity/v3/users/2aa3c1fca03745b7829505384b45db10 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:58.439794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:58.441858 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:58.456006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c3ed6-f68f-4655-98df-7063266a5383 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:58.515077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:58.515860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:58.516173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:58.516450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] PATH_INFO: `/v3/projects/57bdc4f508dc4af1916c7b9d327ea1de` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:58.517663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bdc4f508dc4af1916c7b9d327ea1de)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:58.519590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:58.567070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bdc4f508dc4af1916c7b9d327ea1de', 'request_id': 'req-293bba3e-a61e-4ab8-8475-28642fec822c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.573333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bdc4f508dc4af1916c7b9d327ea1de', 'request_id': 'req-293bba3e-a61e-4ab8-8475-28642fec822c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.581712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bdc4f508dc4af1916c7b9d327ea1de', 'request_id': 'req-293bba3e-a61e-4ab8-8475-28642fec822c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.589904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:58.594323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] The token cache is being invalidate because project 57bdc4f508dc4af1916c7b9d327ea1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:58.594636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.595761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.597701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1124/2240] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:58 2022] DELETE /identity/v3/projects/57bdc4f508dc4af1916c7b9d327ea1de => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:01:58.601974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:58.603687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:58.616619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b3a0-4b9e-4103-956a-90b18e598de0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:58.674524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:58.675448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:58.675730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:58.676025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] PATH_INFO: `/v3/users/021be7076965420fa22de5afa90aa5e6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:58.677305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=021be7076965420fa22de5afa90aa5e6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:58.679240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:58.728882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021be7076965420fa22de5afa90aa5e6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.729535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021be7076965420fa22de5afa90aa5e6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.740145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021be7076965420fa22de5afa90aa5e6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.740750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021be7076965420fa22de5afa90aa5e6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.752417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db68b80b-4673-4e43-9436-d68e1d770073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021be7076965420fa22de5afa90aa5e6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:58.766683 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1117/2241] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:01:58 2022] DELETE /identity/v3/users/021be7076965420fa22de5afa90aa5e6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:59.084692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.086116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.102405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bba3e-a61e-4ab8-8475-28642fec822c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.176490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.177395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.177675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.177953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] PATH_INFO: `/v3/projects/8d1d92371bc74eddb5ba6db7b46bfc10` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.179141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1d92371bc74eddb5ba6db7b46bfc10)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.181214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.231630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1d92371bc74eddb5ba6db7b46bfc10', 'request_id': 'req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:59.237452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1d92371bc74eddb5ba6db7b46bfc10', 'request_id': 'req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:59.246446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1d92371bc74eddb5ba6db7b46bfc10', 'request_id': 'req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:59.259228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] The token cache is being invalidate because project 8d1d92371bc74eddb5ba6db7b46bfc10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:01:59.259576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:59.260317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:01:59.262057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1125/2242] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:01:59 2022] DELETE /identity/v3/projects/8d1d92371bc74eddb5ba6db7b46bfc10 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:59.267955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733985c6-a05b-49c1-bace-67af9f7bb2ea None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.268245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733985c6-a05b-49c1-bace-67af9f7bb2ea None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.268521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733985c6-a05b-49c1-bace-67af9f7bb2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.288797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-733985c6-a05b-49c1-bace-67af9f7bb2ea None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:59.290081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-733985c6-a05b-49c1-bace-67af9f7bb2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.307323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.308814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.335985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:59.355417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1118/2243] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:01:59.365038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.365339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.365611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.371647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782cf6d-f9af-4844-b2dc-73b4ff9d3a20 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.376717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.377368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.377646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.377919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.381159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.386559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:59.388010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.396282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1119/2244] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:59.400452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.409775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9a472-0681-4926-9322-64b3cdc591a2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.415515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.416469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.416922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.417196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.417930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:01:59.418246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.419861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.430589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:01:59.434781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1120/2245] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:01:59.439193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.448300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.450103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.450242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1e2a3-1398-4ea2-ba2b-bc6c4ad8aa04 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.455014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.455908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.456117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.456394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.457088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.458993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.461003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1126/2246] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:01:59.490386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1121/2247] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:01:59 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:01:59.494474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.505064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2597bd-8bb9-479d-b117-04bd8b8b6729 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.562932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.563678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.564016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.564291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.565093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.567179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.590878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1127/2248] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:01:59 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:01:59.592043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.602278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ae82-aa40-4999-b5a8-7839d3537f53 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.607134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.607793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.608117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.608393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.609168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.610925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.622248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1122/2249] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:01:59.626376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.636640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cf1d-a4d9-4ab9-b053-89f32c7b2164 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.641920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cc9d4-1c45-40c8-8714-25f8ab0737bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.642585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cc9d4-1c45-40c8-8714-25f8ab0737bd admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.643145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cc9d4-1c45-40c8-8714-25f8ab0737bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.643145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cc9d4-1c45-40c8-8714-25f8ab0737bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.643966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cc9d4-1c45-40c8-8714-25f8ab0737bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.645516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cc9d4-1c45-40c8-8714-25f8ab0737bd admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.656317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1128/2250] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:01:59.660930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.672033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c2734-07d8-4503-b362-2cab940f3d52 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.678045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.679266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.679553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.679827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] PATH_INFO: `/v3/projects/70838b67c2b14229be10bf5396bc06ed/users/126d80fa459a48e6826993acee2db37a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.696416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70838b67c2b14229be10bf5396bc06ed, user_id=126d80fa459a48e6826993acee2db37a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.699182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.713373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1123/2251] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:01:59 2022] PUT /identity/v3/projects/70838b67c2b14229be10bf5396bc06ed/users/126d80fa459a48e6826993acee2db37a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:01:59.724409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.724697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.724975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.752037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] MFA Rules not processed for user `126d80fa459a48e6826993acee2db37a`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:01:59.753276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.822675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1129/2252] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:01:59.835003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:01:59.837355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:01:59.902663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029664-1138-436e-bcbd-0f7bf24f45d5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:01:59.907054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:01:59.907697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:01:59.908008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:01:59.908284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:01:59.917304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:01:59.918747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:01:59.927491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1124/2253] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:00.432276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:00.433960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:00.496220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5901669-abf9-4fcd-b486-6f2159327c6e None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:00.499818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:00.500522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:00.500805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:00.501076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:00.502866 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:00.510168 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:00.571077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:00.572529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2856bf-82a3-4663-aeec-d01f3ad79f0a service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:00.582147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1130/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:05.237927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:05.240775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:05.337714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfc2e5-0f33-4348-b392-6a93ff917f3e service glance] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:05.343166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:05.344272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:05.344644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:05.344982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] PATH_INFO: `/v3/users/25d5cecef695490ebf4319f282496220` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:05.346291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d5cecef695490ebf4319f282496220)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:05.348395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:05.366194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:05.405421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d5cecef695490ebf4319f282496220'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.406204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d5cecef695490ebf4319f282496220'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.415596 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d5cecef695490ebf4319f282496220'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.416237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d5cecef695490ebf4319f282496220'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.428708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d5cecef695490ebf4319f282496220'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.455306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1125/2255] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:05 2022] DELETE /identity/v3/users/25d5cecef695490ebf4319f282496220 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:05.459930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:05.463352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:05.464193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:05.482574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:05.483775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:05.560753 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1131/2256] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:02:05.571831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:05.573506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:05.637436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1181556-e8a2-430c-8a47-a2db3794e58b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:05.641686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b29ff-7e1c-4e67-a3b5-835fa8e44ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:05.642341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b29ff-7e1c-4e67-a3b5-835fa8e44ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:05.642627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b29ff-7e1c-4e67-a3b5-835fa8e44ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:05.642908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b29ff-7e1c-4e67-a3b5-835fa8e44ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:05.661697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b29ff-7e1c-4e67-a3b5-835fa8e44ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:05.663158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b29ff-7e1c-4e67-a3b5-835fa8e44ac2 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:05.675572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1126/2257] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:05.880893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:05.883215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:05.901166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabc372-a0c4-4499-abda-cc39c82a5e2a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:05.907354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:05.908457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:05.908795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:05.909134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] PATH_INFO: `/v3/users/b161bf44482c4687ae987272dcd59496` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:05.910407 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b161bf44482c4687ae987272dcd59496)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:05.912451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:05.963009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b161bf44482c4687ae987272dcd59496'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.964019 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b161bf44482c4687ae987272dcd59496'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.964824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b161bf44482c4687ae987272dcd59496'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.973894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b161bf44482c4687ae987272dcd59496'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:05.986834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b161bf44482c4687ae987272dcd59496'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.002384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1132/2258] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:05 2022] DELETE /identity/v3/users/b161bf44482c4687ae987272dcd59496 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:06.007363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.007753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.008242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.021922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.023138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.029055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:06.030575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.045595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:06.107339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9906f7f-2290-43a0-8920-c91ecc3f0e9d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.118348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.121386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.121907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.122891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] PATH_INFO: `/v3/projects/31468a1725ed43f6b93208484155d1c6` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.125981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31468a1725ed43f6b93208484155d1c6)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.128630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.129737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1127/2259] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:02:06.151927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.151927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.199368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31468a1725ed43f6b93208484155d1c6', 'request_id': 'req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.201021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:06.206200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31468a1725ed43f6b93208484155d1c6', 'request_id': 'req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.217818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31468a1725ed43f6b93208484155d1c6', 'request_id': 'req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.223347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb40541-7753-4571-835c-0946e5d26ade None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.233093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] The token cache is being invalidate because project 31468a1725ed43f6b93208484155d1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:06.233471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.234217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7fa40c7c-0618-46f4-bf87-4bdf64817cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.235893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1133/2260] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:06 2022] DELETE /identity/v3/projects/31468a1725ed43f6b93208484155d1c6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:02:06.241267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4358ff0-1c52-45ea-9266-9f1cef0c8ece None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.241609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4358ff0-1c52-45ea-9266-9f1cef0c8ece None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.241940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4358ff0-1c52-45ea-9266-9f1cef0c8ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.260902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b4358ff0-1c52-45ea-9266-9f1cef0c8ece None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:06.262180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b4358ff0-1c52-45ea-9266-9f1cef0c8ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.285543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.286643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.286731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.287128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.296207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.361544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1134/2261] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:06.370500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.370918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.371303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.386950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.388618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.391557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:06.392966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.398828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1128/2262] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:02:06.401836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1135/2263] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:02:06.406101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.417276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0616778-46af-4a69-a89d-69e28687d8fd service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.423452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.423978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.424233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.424537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.425437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:02:06.425694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.427336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.441716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1129/2264] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:02:06.447225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.460098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a6046-87ff-4f7c-b8c9-3aba7f37c99c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.465761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.466473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.467127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.467127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.467805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.469791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.501309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1136/2265] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:02:06 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:06.505368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.515501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b933e06-873e-4c43-b3f5-08ba90aa7d71 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.583802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.584691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.585044 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.585420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.586160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.588265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.608242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1130/2266] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:02:06 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:02:06.612466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.622873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c431d-a3f5-41cc-9ba5-86978e873e13 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.627550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.628250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.628529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.628800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.629530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.631127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.641701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1137/2267] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:06.645617 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.656509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689103-97e6-4ad0-bbf3-a57f67dbcaaa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.661471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.662147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.662412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.662806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.663586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.665592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.676900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1131/2268] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:06.677301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.679301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.680944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.691097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf68b0b-4777-4ea2-990d-bc1661a59382 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.697098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca476372-861d-47f1-955c-046803194a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.698002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca476372-861d-47f1-955c-046803194a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.698283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca476372-861d-47f1-955c-046803194a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.698567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca476372-861d-47f1-955c-046803194a05 admin admin] PATH_INFO: `/v3/projects/cdee54b3d0784c2187129aa2473da0c3/users/f41a5f68414e4609a982dd43e09bcba3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.701636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670231d1-091b-41bb-82b2-c15c52684d86 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.701941 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:06.708077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.708998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.709280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.709554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] PATH_INFO: `/v3/projects/2070ae6daaac45af9a3dda314428077f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.710872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2070ae6daaac45af9a3dda314428077f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.713152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.714995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:06.716951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca476372-861d-47f1-955c-046803194a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdee54b3d0784c2187129aa2473da0c3, user_id=f41a5f68414e4609a982dd43e09bcba3, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.719622 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca476372-861d-47f1-955c-046803194a05 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.734227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1132/2269] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:06 2022] PUT /identity/v3/projects/cdee54b3d0784c2187129aa2473da0c3/users/f41a5f68414e4609a982dd43e09bcba3/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:02:06.744335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.744618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.744893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.766918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2070ae6daaac45af9a3dda314428077f', 'request_id': 'req-8da8f544-c295-4e9b-a132-d840abced314'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.771942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] MFA Rules not processed for user `f41a5f68414e4609a982dd43e09bcba3`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:06.773206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.773352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2070ae6daaac45af9a3dda314428077f', 'request_id': 'req-8da8f544-c295-4e9b-a132-d840abced314'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.785423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2070ae6daaac45af9a3dda314428077f', 'request_id': 'req-8da8f544-c295-4e9b-a132-d840abced314'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.801475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] The token cache is being invalidate because project 2070ae6daaac45af9a3dda314428077f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:06.801770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.802493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.804098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1138/2270] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:06 2022] DELETE /identity/v3/projects/2070ae6daaac45af9a3dda314428077f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:06.808029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.809740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.861532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1133/2271] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:02:06.872307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:06.873677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:06.881408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f544-c295-4e9b-a132-d840abced314 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.886908 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.887794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.888110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.888381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] PATH_INFO: `/v3/users/b4aed351fa9145bfbc7403273cab561e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.889603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4aed351fa9145bfbc7403273cab561e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.891750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.948024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4abaa-d7f9-49b3-b933-79a14d864368 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:06.954313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:06.954964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:06.955240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:06.955512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:06.958632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4aed351fa9145bfbc7403273cab561e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.959377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4aed351fa9145bfbc7403273cab561e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.960136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4aed351fa9145bfbc7403273cab561e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.965420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:06.967468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:06.970340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4aed351fa9145bfbc7403273cab561e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:06.980490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1134/2272] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:02:06.988140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4aed351fa9145bfbc7403273cab561e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.002621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1139/2273] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:06 2022] DELETE /identity/v3/users/b4aed351fa9145bfbc7403273cab561e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:02:07.367945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:07.381192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19cc4a-9517-47e5-b3b6-071f6c116fbd service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:07.439574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:07.440912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:07.441328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:07.441733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] PATH_INFO: `/v3/projects/9008f7b0e7a04481b257d4f8920e7919` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:07.443691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9008f7b0e7a04481b257d4f8920e7919)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:07.447259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:07.473492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:07.502525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9008f7b0e7a04481b257d4f8920e7919', 'request_id': 'req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.508147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9008f7b0e7a04481b257d4f8920e7919', 'request_id': 'req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.514678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9008f7b0e7a04481b257d4f8920e7919', 'request_id': 'req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.528592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] The token cache is being invalidate because project 9008f7b0e7a04481b257d4f8920e7919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:07.528863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.529782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.531647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1135/2274] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:07 2022] DELETE /identity/v3/projects/9008f7b0e7a04481b257d4f8920e7919 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:07.536463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:07.538357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:07.552401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1393-fa20-46c3-b2c4-29036d3b5231 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:07.557703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:07.613128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:07.614008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:07.614291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:07.614568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] PATH_INFO: `/v3/users/8536151b153d4223992e08de1ad752d3` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:07.616424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8536151b153d4223992e08de1ad752d3)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:07.618385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:07.680498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8536151b153d4223992e08de1ad752d3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.681276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8536151b153d4223992e08de1ad752d3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.681918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8536151b153d4223992e08de1ad752d3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.691955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8536151b153d4223992e08de1ad752d3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.707136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7b930170-ecce-4305-88ee-2c06346c29ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8536151b153d4223992e08de1ad752d3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:07.720147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1140/2275] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:07 2022] DELETE /identity/v3/users/8536151b153d4223992e08de1ad752d3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:08.025557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.027374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:08.051553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fe2da-9b70-4dbc-8800-9cf3bd1ad76d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.111140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.112017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.112273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.112696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] PATH_INFO: `/v3/projects/9686d7de78ce4b1abd29547aedbc4605` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.113862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9686d7de78ce4b1abd29547aedbc4605)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.115778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.182335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9686d7de78ce4b1abd29547aedbc4605', 'request_id': 'req-a4aab9b9-5220-4b06-8e71-5cb537187c74'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:08.191052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9686d7de78ce4b1abd29547aedbc4605', 'request_id': 'req-a4aab9b9-5220-4b06-8e71-5cb537187c74'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:08.198193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9686d7de78ce4b1abd29547aedbc4605', 'request_id': 'req-a4aab9b9-5220-4b06-8e71-5cb537187c74'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:08.214299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] The token cache is being invalidate because project 9686d7de78ce4b1abd29547aedbc4605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:08.214575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:08.215242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-a4aab9b9-5220-4b06-8e71-5cb537187c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:08.216839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1136/2276] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:08 2022] DELETE /identity/v3/projects/9686d7de78ce4b1abd29547aedbc4605 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:08.222070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.222420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.222752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.242021 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:08.243690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:08.320077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1141/2277] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:08.390370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.390997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.392045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.420631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:08.422194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:08.432736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1137/2278] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:08.436606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.449212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479471f-6006-47a9-abce-b0e228a3bb3c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.460347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.461346 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.461680 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.462073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.463034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:02:08.463509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.465232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.480057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1142/2279] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:02:08.483686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.498130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1132dd5-cc98-4b7b-8233-e33165a17df3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.509299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.510306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.510867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.511613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.512330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.514420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.547915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1138/2280] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:02:08 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:08.551089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.562013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcbf4c-b594-497a-ac22-df8c7192f25f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.580986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:08.638838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.639008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.639283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.639601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.640457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.642461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.671826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1143/2281] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:02:08 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:08.672796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.684607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8d96-101b-4804-b41c-43b124cb9833 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.692693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.693504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.694006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.694580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.695574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.699001 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.734505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:08.755947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1139/2282] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:08.756541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.773651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0d0bf-43b2-4fd9-a3a0-a15d95fbf7d1 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.787664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dedb07-b300-4cd0-a5ee-ed1c631f6e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.788412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dedb07-b300-4cd0-a5ee-ed1c631f6e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.789343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dedb07-b300-4cd0-a5ee-ed1c631f6e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.795504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dedb07-b300-4cd0-a5ee-ed1c631f6e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.797703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dedb07-b300-4cd0-a5ee-ed1c631f6e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.799537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dedb07-b300-4cd0-a5ee-ed1c631f6e2b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.828067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1144/2283] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:08.830564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:08.841823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b343b-9c6a-438c-958e-0864a54accb4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:08.851025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:08.852056 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.852495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.852773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] PATH_INFO: `/v3/projects/d589375d668e413d839c63b0c3d4f104/users/aa782d84840842ef983ed058379d5626/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.869409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d589375d668e413d839c63b0c3d4f104, user_id=aa782d84840842ef983ed058379d5626, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:08.872476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:08.888378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1140/2284] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:08 2022] PUT /identity/v3/projects/d589375d668e413d839c63b0c3d4f104/users/aa782d84840842ef983ed058379d5626/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:08.893495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b7bc9-d73a-4fdc-8396-6aee4dc948e5 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:08.895899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b7bc9-d73a-4fdc-8396-6aee4dc948e5 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:08.895899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b7bc9-d73a-4fdc-8396-6aee4dc948e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:08.941551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a63b7bc9-d73a-4fdc-8396-6aee4dc948e5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:08.941551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a63b7bc9-d73a-4fdc-8396-6aee4dc948e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:09.059938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1145/2285] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:09.061014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:09.062732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:09.128297 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e10b3-dcc1-42a6-bab8-06d68654c88e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:09.132819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:09.133624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:09.133909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:09.134224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:09.145672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:09.147110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:09.157846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1141/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:02:09.999811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81625a-2464-4c4e-a7d6-51039550b5e6 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:10.000293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81625a-2464-4c4e-a7d6-51039550b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:10.000493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81625a-2464-4c4e-a7d6-51039550b5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:10.040171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3f81625a-2464-4c4e-a7d6-51039550b5e6 None None] MFA Rules not processed for user `64fe0b9043ad479284163d04a90b842f`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:10.041398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3f81625a-2464-4c4e-a7d6-51039550b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:10.113975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1146/2287] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:10.257678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:10.260551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:10.274209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7614a9-758d-4f27-ae0b-1094f4727b94 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:10.280498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:10.284525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:10.285084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:10.286263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] PATH_INFO: `/v3/users/f41a5f68414e4609a982dd43e09bcba3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:10.288263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41a5f68414e4609a982dd43e09bcba3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:10.290849 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:10.348576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41a5f68414e4609a982dd43e09bcba3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:10.349780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41a5f68414e4609a982dd43e09bcba3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:10.372188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41a5f68414e4609a982dd43e09bcba3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:10.373582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41a5f68414e4609a982dd43e09bcba3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:10.396062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41a5f68414e4609a982dd43e09bcba3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:10.410794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1142/2288] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:10 2022] DELETE /identity/v3/users/f41a5f68414e4609a982dd43e09bcba3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:10.414741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:10.415050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:10.415433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:10.433154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:10.434463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:10.473347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:10.527245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1147/2289] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:10.536916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:10.538406 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:10.617089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:10.622641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d32c-5a6d-4e3e-8996-5d4316b0cc03 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:10.628815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183aa0b-f007-4973-8b33-932591db62fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:10.629808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183aa0b-f007-4973-8b33-932591db62fd service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:10.630096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183aa0b-f007-4973-8b33-932591db62fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:10.630365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183aa0b-f007-4973-8b33-932591db62fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:10.641581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183aa0b-f007-4973-8b33-932591db62fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:10.643109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183aa0b-f007-4973-8b33-932591db62fd service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:10.654830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1143/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:11.010155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.012623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:11.054745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469884ed-f28e-456e-88b2-98bd679f6c48 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.061237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.062567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.062855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.063130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] PATH_INFO: `/v3/projects/cdee54b3d0784c2187129aa2473da0c3` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.064438 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdee54b3d0784c2187129aa2473da0c3)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.066345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.144050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdee54b3d0784c2187129aa2473da0c3', 'request_id': 'req-cec85c65-dcc7-4681-875b-d833b7ce4fed'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:11.151088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdee54b3d0784c2187129aa2473da0c3', 'request_id': 'req-cec85c65-dcc7-4681-875b-d833b7ce4fed'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:11.161028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdee54b3d0784c2187129aa2473da0c3', 'request_id': 'req-cec85c65-dcc7-4681-875b-d833b7ce4fed'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:11.176553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] The token cache is being invalidate because project cdee54b3d0784c2187129aa2473da0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:11.176553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:11.177226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-cec85c65-dcc7-4681-875b-d833b7ce4fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:11.179381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1148/2291] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:11 2022] DELETE /identity/v3/projects/cdee54b3d0784c2187129aa2473da0c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:11.185061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.185287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.185635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.209681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:11.211655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:11.335671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1144/2292] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:11.344003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.344296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.344575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.380682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:11.382309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:11.407205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1149/2293] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:11.415863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.427370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd845ac-84ab-441b-938a-6e6baa5b8b70 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.432964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.433646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.433928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.434206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.434965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:02:11.435447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.437112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.451637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1145/2294] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:02:11.455663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.466940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc6e6-b91c-4da8-9fee-258ce8cf07ac None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.479526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.480526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.488161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.489052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.489841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.491812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.529137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1150/2295] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:02:11 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:11.533224 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.546846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437548d-421b-4cde-ad5f-52fc2b1e02ee admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.664555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.665310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.665594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.665867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.666626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.668799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.695318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1146/2296] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:02:11 2022] POST /identity/v3/users => generated 284 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:11.700053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.711001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dc491-909e-4018-b5bc-c43952bdba9c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.718948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.719635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.719928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.720491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.720925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.722720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.740476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1151/2297] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:11.742095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.755915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b84299-adbb-4735-96e0-1c4e43f9399e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.761721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692d08c-57d3-46ef-b5b3-d82c6ed5e66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.762579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692d08c-57d3-46ef-b5b3-d82c6ed5e66c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.763655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692d08c-57d3-46ef-b5b3-d82c6ed5e66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.763655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692d08c-57d3-46ef-b5b3-d82c6ed5e66c admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.764203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692d08c-57d3-46ef-b5b3-d82c6ed5e66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.766098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692d08c-57d3-46ef-b5b3-d82c6ed5e66c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.779470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1147/2298] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:11.783775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:11.798696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db751f-8f8d-436d-acdd-8621ac2104f5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:11.806337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:11.807584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.808236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.808852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] PATH_INFO: `/v3/projects/33b63280cc3947da8d0e57634b9fa2b9/users/413fe1395d684077bbeac2251c62f245/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.828817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b63280cc3947da8d0e57634b9fa2b9, user_id=413fe1395d684077bbeac2251c62f245, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:11.831734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:11.855922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1152/2299] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:11 2022] PUT /identity/v3/projects/33b63280cc3947da8d0e57634b9fa2b9/users/413fe1395d684077bbeac2251c62f245/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:02:11.863999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed355f-333b-46fa-b424-202a515674ff None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:11.864285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed355f-333b-46fa-b424-202a515674ff None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:11.864558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed355f-333b-46fa-b424-202a515674ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:11.924521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e9ed355f-333b-46fa-b424-202a515674ff None None] MFA Rules not processed for user `413fe1395d684077bbeac2251c62f245`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:11.925932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e9ed355f-333b-46fa-b424-202a515674ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:11.998871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:12.030320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1148/2300] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:12.046416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:12.048567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:12.212707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8f6e1-495d-4e4e-9dd9-27e538bafa35 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:12.216714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:12.217655 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:12.218134 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:12.218575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:12.228757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:12.230264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:12.248533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1153/2301] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:14.590719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:14.591319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:14.591319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:14.645724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] MFA Rules not processed for user `aa782d84840842ef983ed058379d5626`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:14.646953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:14.750286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1149/2302] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:14.755209 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:14.756922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:14.768999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:14.843470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568cc5b-f5bf-4f5d-b80c-6ac7b29d0626 service glance] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:14.848432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:14.849107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:14.849517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:14.849797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:14.850512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:14.852716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:14.885977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1154/2303] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:02:14 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:14.889877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:14.903195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dace3f-70ce-4ba6-ab60-1f1e7543e9d6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:14.977891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:14.978851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:14.979157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:14.979540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:14.980346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:14.983764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:15.014024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1150/2304] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:02:14 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:15.018291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:15.031268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa567714-d3be-48bf-a953-c36861f6ed15 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:15.040818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:15.041833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:15.042201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:15.042537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:15.046552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:15.048463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:15.060605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1155/2305] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:15.064858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:15.076121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5aa36-5c7d-4375-9fca-56f66b10966c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:15.088751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:15.090500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:15.090500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:15.090500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] PATH_INFO: `/v3/projects/7fb082c5eb6f43d7b0aa99b2a003f518/users/ccce04f4f436432bbacc0a66ba6a52a9/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:15.110638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb082c5eb6f43d7b0aa99b2a003f518, user_id=ccce04f4f436432bbacc0a66ba6a52a9, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:15.113846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:15.131819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1151/2306] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:15 2022] PUT /identity/v3/projects/7fb082c5eb6f43d7b0aa99b2a003f518/users/ccce04f4f436432bbacc0a66ba6a52a9/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:02:15.136214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:15.149025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e75ba-92f9-4725-b4c5-d16c8ff2a14e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:15.238376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d38a-c119-42ff-8ce0-2d2d1dfc5023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:15.239164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d38a-c119-42ff-8ce0-2d2d1dfc5023 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:15.239519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d38a-c119-42ff-8ce0-2d2d1dfc5023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:15.239862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d38a-c119-42ff-8ce0-2d2d1dfc5023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:15.240706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d38a-c119-42ff-8ce0-2d2d1dfc5023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:15.243188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d38a-c119-42ff-8ce0-2d2d1dfc5023 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:15.257128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1156/2307] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:15.261262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:15.271807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a594330-bba4-4b80-933e-615237a41486 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:15.287609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:15.289269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:15.291414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:15.292121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] PATH_INFO: `/v3/projects/7fb082c5eb6f43d7b0aa99b2a003f518/users/ccce04f4f436432bbacc0a66ba6a52a9/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:15.298416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb082c5eb6f43d7b0aa99b2a003f518, user_id=ccce04f4f436432bbacc0a66ba6a52a9, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:15.303008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:15.323946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1152/2308] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:15 2022] PUT /identity/v3/projects/7fb082c5eb6f43d7b0aa99b2a003f518/users/ccce04f4f436432bbacc0a66ba6a52a9/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:21.566711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983078bd-c125-4ced-8999-cff3e39d69c3 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:21.567315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983078bd-c125-4ced-8999-cff3e39d69c3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:21.567315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983078bd-c125-4ced-8999-cff3e39d69c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:21.600102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-983078bd-c125-4ced-8999-cff3e39d69c3 None None] MFA Rules not processed for user `ccce04f4f436432bbacc0a66ba6a52a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:21.601336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-983078bd-c125-4ced-8999-cff3e39d69c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:21.674408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:21.680878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1157/2309] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:21.693747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:21.696938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:21.734803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:21.769798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823fa93-ae12-4f49-b6f6-7d66e6e85b41 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:21.773801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:21.774456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:21.774736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:21.775014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:21.846192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:21.847668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:21.860153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1153/2310] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:22.016231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c85477-6e84-4557-ade6-b994dfc43e8d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:22.016707 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c85477-6e84-4557-ade6-b994dfc43e8d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:22.017282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c85477-6e84-4557-ade6-b994dfc43e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:22.040867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-35c85477-6e84-4557-ade6-b994dfc43e8d None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:22.042825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-35c85477-6e84-4557-ade6-b994dfc43e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:22.129533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1158/2311] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:02:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:22.193546 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:22.195433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:22.298761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ece75f-8909-4c64-84b7-42419891b39d service glance] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:22.305812 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:22.306936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:22.307403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:22.307852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:22.322767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:22.324448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:22.333319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1154/2312] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:02:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:02:32.123442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=26 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:32.331373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=67 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:33.038917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09252e-8a21-43e8-875e-37e0095c45d2 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:33.039649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09252e-8a21-43e8-875e-37e0095c45d2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:33.040228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09252e-8a21-43e8-875e-37e0095c45d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:33.073256 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6d09252e-8a21-43e8-875e-37e0095c45d2 None None] MFA Rules not processed for user `aa782d84840842ef983ed058379d5626`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:33.077281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6d09252e-8a21-43e8-875e-37e0095c45d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:33.084501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1159/2313] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:34.225584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:34.227257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:34.299239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e94c8-7cf0-466a-ac54-d85b5c5aadbf service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:34.305358 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:34.306387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:34.306661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:34.307047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] PATH_INFO: `/v3/users/126d80fa459a48e6826993acee2db37a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:34.308272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126d80fa459a48e6826993acee2db37a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:34.310180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:34.366110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126d80fa459a48e6826993acee2db37a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:34.366751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126d80fa459a48e6826993acee2db37a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:34.375726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126d80fa459a48e6826993acee2db37a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:34.376903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126d80fa459a48e6826993acee2db37a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:34.389280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126d80fa459a48e6826993acee2db37a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:34.402237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1155/2314] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:34 2022] DELETE /identity/v3/users/126d80fa459a48e6826993acee2db37a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:34.407225 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:34.407770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:34.407906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:34.433546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:34.434888 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:34.498370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1160/2315] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:34.508451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:34.510242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:34.571906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15534-14e7-4d73-8729-19b79e339b42 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:34.578191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a077e91-bffe-414e-af2b-8e4369dfc9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:34.578867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a077e91-bffe-414e-af2b-8e4369dfc9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:34.579144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a077e91-bffe-414e-af2b-8e4369dfc9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:34.579420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a077e91-bffe-414e-af2b-8e4369dfc9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:34.589919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a077e91-bffe-414e-af2b-8e4369dfc9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:34.591404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a077e91-bffe-414e-af2b-8e4369dfc9ab service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:34.601594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1156/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:34.918508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:34.920090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:34.932949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881caa35-2c49-46a9-9e48-926c14f71b76 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:34.938250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:34.939147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:34.939429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:34.939702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] PATH_INFO: `/v3/projects/70838b67c2b14229be10bf5396bc06ed` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:34.941018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70838b67c2b14229be10bf5396bc06ed)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:34.943170 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:34.995257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70838b67c2b14229be10bf5396bc06ed', 'request_id': 'req-92a26326-b2f3-417d-986d-2594c6f98b9c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:35.000711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70838b67c2b14229be10bf5396bc06ed', 'request_id': 'req-92a26326-b2f3-417d-986d-2594c6f98b9c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:35.009088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70838b67c2b14229be10bf5396bc06ed', 'request_id': 'req-92a26326-b2f3-417d-986d-2594c6f98b9c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:35.019860 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] The token cache is being invalidate because project 70838b67c2b14229be10bf5396bc06ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:35.020176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:35.020802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-92a26326-b2f3-417d-986d-2594c6f98b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:35.022428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1161/2317] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:34 2022] DELETE /identity/v3/projects/70838b67c2b14229be10bf5396bc06ed => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:35.027415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.027673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.027978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.047630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:35.048882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:35.067747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:35.116713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1157/2318] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:35.125628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.125919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.126197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.146453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:35.147131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:35.156531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1162/2319] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:35.160865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.171384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d7d29-e123-4cae-a46e-2536159d68a0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.176428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.177086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.177366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.177641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.178401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:02:35.178847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.180740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.192416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1158/2320] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:02:35.197425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.207079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d59125-cbde-4079-9ece-287915b92c86 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.212432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.213186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.213473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.213741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.214437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.216371 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.244582 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1163/2321] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:02:35 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:35.248696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.258804 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f7343-bc67-47fb-95b2-80eba428ba00 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.324401 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.325079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.325353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.325626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.326318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.328306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.353434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1159/2322] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:02:35 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:35.357622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.368359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2f8b-dda5-4650-b3c3-17c0e83e5dcf admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.373130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.373796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.374092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.374792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.375367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.377031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.388065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1164/2323] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:35.392150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.403160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d80f0-64e9-40e8-ba12-052dd998bb86 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.408158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07bb7-f5b3-474f-bc7c-18eb586d944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.408852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07bb7-f5b3-474f-bc7c-18eb586d944e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.409130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07bb7-f5b3-474f-bc7c-18eb586d944e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.409404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07bb7-f5b3-474f-bc7c-18eb586d944e admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.410136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07bb7-f5b3-474f-bc7c-18eb586d944e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.412121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07bb7-f5b3-474f-bc7c-18eb586d944e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.422832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1160/2324] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:35.427044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.436939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb35b-7a9e-4239-9a70-5512230fdc83 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.441637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.442520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.442796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.443063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] PATH_INFO: `/v3/projects/8de5ed099cad4e6aa38f12d16cfb9615/users/957f396e32bc48c0866a53122122bc7a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.460010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de5ed099cad4e6aa38f12d16cfb9615, user_id=957f396e32bc48c0866a53122122bc7a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.462705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.477467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1165/2325] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:35 2022] PUT /identity/v3/projects/8de5ed099cad4e6aa38f12d16cfb9615/users/957f396e32bc48c0866a53122122bc7a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:35.483057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15826765-0c75-48b5-8fe5-fe5e582e4346 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.483421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15826765-0c75-48b5-8fe5-fe5e582e4346 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.483841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15826765-0c75-48b5-8fe5-fe5e582e4346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.502598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-15826765-0c75-48b5-8fe5-fe5e582e4346 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:35.503820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-15826765-0c75-48b5-8fe5-fe5e582e4346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:35.566967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1161/2326] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:35.577325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:35.579078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:35.596086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:35.645290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94c228-400e-468f-bfae-cb651afe16ab admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:35.649379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:35.650037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:35.650579 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:35.650690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:35.661751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:35.662441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:35.672960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1166/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:36.365317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3fce2-99dc-4f4f-9a43-a36caab868ef None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:36.365954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3fce2-99dc-4f4f-9a43-a36caab868ef None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:36.365954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3fce2-99dc-4f4f-9a43-a36caab868ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:36.397802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-29a3fce2-99dc-4f4f-9a43-a36caab868ef None None] MFA Rules not processed for user `957f396e32bc48c0866a53122122bc7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:36.399032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-29a3fce2-99dc-4f4f-9a43-a36caab868ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:36.417452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:36.490471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1162/2328] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:36.507385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:36.508924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:36.592965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d5fa9-b56d-4b8c-bacb-51999d6ed7d4 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:36.597378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d2bbd-a285-49ef-9a89-70e49946c7c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:36.598467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d2bbd-a285-49ef-9a89-70e49946c7c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:36.599113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d2bbd-a285-49ef-9a89-70e49946c7c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:36.599693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d2bbd-a285-49ef-9a89-70e49946c7c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:36.611228 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d2bbd-a285-49ef-9a89-70e49946c7c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:36.612782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d2bbd-a285-49ef-9a89-70e49946c7c4 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:36.623852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1167/2329] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:37.998884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:37.999445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:37.999445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:38.018371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] MFA Rules not processed for user `957f396e32bc48c0866a53122122bc7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:38.019562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:38.028710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1163/2330] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:38.258196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175db84-5bb2-4b00-965f-ecb1752d7cf3 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:38.258657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175db84-5bb2-4b00-965f-ecb1752d7cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:38.258761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175db84-5bb2-4b00-965f-ecb1752d7cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:38.282820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6175db84-5bb2-4b00-965f-ecb1752d7cf3 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:38.284073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6175db84-5bb2-4b00-965f-ecb1752d7cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:38.346593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1168/2331] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:02:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:38.359753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:38.361183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:38.436761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb2276-1539-4305-acdf-508fe90142b5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:38.441861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:38.442509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:38.442834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:38.443066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:38.452828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:38.455809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:38.470331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1164/2332] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:02:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:02:42.662452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfc585-41e5-4fe6-80d0-b93d415328e2 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:42.663258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfc585-41e5-4fe6-80d0-b93d415328e2 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:42.663258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfc585-41e5-4fe6-80d0-b93d415328e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:42.713199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-d4dfc585-41e5-4fe6-80d0-b93d415328e2 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:42.714811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d4dfc585-41e5-4fe6-80d0-b93d415328e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:42.730776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1169/2333] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:02:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:42.745447 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:42.758960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbaf0b4-a040-430a-8763-841f92d4fa5a service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:42.764840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:42.765637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:42.766018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:42.767428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:42.783320 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:42.800324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:42.825122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1165/2334] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:02:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:02:45.625814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b06300-02c1-4734-b9c3-fb084506baa4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:45.626387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b06300-02c1-4734-b9c3-fb084506baa4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:45.626387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b06300-02c1-4734-b9c3-fb084506baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:45.681387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-95b06300-02c1-4734-b9c3-fb084506baa4 None None] MFA Rules not processed for user `4e9ad2c4909c49a6b3df8ece333e70b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:45.684982 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-95b06300-02c1-4734-b9c3-fb084506baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:45.839541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1170/2335] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:52.816281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=60 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:55.534247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:55.536608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:55.791784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7eff1-3a92-4e4d-9ab5-b220c7ae2e88 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:55.802952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:55.805602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:55.806513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:55.806800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] PATH_INFO: `/v3/users/413fe1395d684077bbeac2251c62f245` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:55.809477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] RBAC: Authorizing `identity:delete_user(user_id=413fe1395d684077bbeac2251c62f245)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:55.813649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:55.822936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:55.953888 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413fe1395d684077bbeac2251c62f245'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:55.955092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413fe1395d684077bbeac2251c62f245'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:55.971394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413fe1395d684077bbeac2251c62f245'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:55.972347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413fe1395d684077bbeac2251c62f245'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:55.992996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413fe1395d684077bbeac2251c62f245'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:56.020009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1166/2336] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:02:55 2022] DELETE /identity/v3/users/413fe1395d684077bbeac2251c62f245 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:56.025060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:56.025327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:56.025614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:56.111561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:56.113627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:56.237931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1171/2337] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:56.289958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:56.289958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:56.463798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:56.503500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8c7de-a6f2-490d-b905-79e5abb4812f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:56.508707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090c5b2-3296-4720-8d41-fc0c0e77de90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:56.509647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090c5b2-3296-4720-8d41-fc0c0e77de90 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:56.510189 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090c5b2-3296-4720-8d41-fc0c0e77de90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:56.510561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090c5b2-3296-4720-8d41-fc0c0e77de90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:56.529151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090c5b2-3296-4720-8d41-fc0c0e77de90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:56.532377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090c5b2-3296-4720-8d41-fc0c0e77de90 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:56.555710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1167/2338] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:57.507369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:57.509518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:57.527560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485176bd-02af-4a02-ae4d-8643a2e932dd None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:57.534009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:57.536054 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:57.537451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:57.538410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] PATH_INFO: `/v3/projects/33b63280cc3947da8d0e57634b9fa2b9` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:57.542105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b63280cc3947da8d0e57634b9fa2b9)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:57.544623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:57.620213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b63280cc3947da8d0e57634b9fa2b9', 'request_id': 'req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:57.628028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b63280cc3947da8d0e57634b9fa2b9', 'request_id': 'req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:57.641307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b63280cc3947da8d0e57634b9fa2b9', 'request_id': 'req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:57.668608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] The token cache is being invalidate because project 33b63280cc3947da8d0e57634b9fa2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:02:57.668918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:57.669615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3ff6f9d8-3270-43e9-9b36-ead48d15b29d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:02:57.671612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1172/2339] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:02:57 2022] DELETE /identity/v3/projects/33b63280cc3947da8d0e57634b9fa2b9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:02:57.676820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:57.677305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:57.677536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:57.721357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:57.722635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:57.784924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1168/2340] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:02:57.792758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:57.793048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:57.794128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:57.814648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:57.815865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:57.843776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1173/2341] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:57.848248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:57.859141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55831f5-98ad-4a7f-9982-0596beb54aa6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:57.863805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:57.864591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:57.865003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:57.865467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:57.866281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:02:57.866649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:57.868342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:57.882248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1169/2342] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:02:57.886576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:57.896590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ee2de-7c3a-4a48-aaa2-171559482e9c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:57.901463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:57.902135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:57.902423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:57.902692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:57.903389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:57.905854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:57.937849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1174/2343] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:02:57 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:02:57.941993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:57.952402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26c81-48ad-4413-aae3-9b3dba9c744e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:58.013107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:58.013788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.014068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.014339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:58.015051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:58.017032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:58.043963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1170/2344] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:02:57 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:02:58.045175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:58.060034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f3cc4-acfa-4edd-b704-70aed04b2d56 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:58.065084 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:58.065746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.065989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.066269 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:58.067003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:58.069148 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:58.080467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1175/2345] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:58.084864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:58.095221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c2fc9-147b-40ac-b6ee-6fb0dd2fa074 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:58.100604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3f47-218a-4c46-ac6a-1e41d1a66242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:58.101446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3f47-218a-4c46-ac6a-1e41d1a66242 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.101962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3f47-218a-4c46-ac6a-1e41d1a66242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.102433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3f47-218a-4c46-ac6a-1e41d1a66242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:58.103242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d3f47-218a-4c46-ac6a-1e41d1a66242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:58.105108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d3f47-218a-4c46-ac6a-1e41d1a66242 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:58.116129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1171/2346] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:02:58.120332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:58.130694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf604c3e-0d0d-41e4-be7e-2f72e13159e0 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:58.135580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:58.136552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.136827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.137086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] PATH_INFO: `/v3/projects/14cadb2391354f1f8b66756c7f087d60/users/3f3e956f03cd457c91c0bfadd517dbaa/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:58.153358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14cadb2391354f1f8b66756c7f087d60, user_id=3f3e956f03cd457c91c0bfadd517dbaa, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:58.156041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:58.170192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1176/2347] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:02:58 2022] PUT /identity/v3/projects/14cadb2391354f1f8b66756c7f087d60/users/3f3e956f03cd457c91c0bfadd517dbaa/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:02:58.181699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6803b-d114-4107-afb4-75ad156ee481 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.181990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6803b-d114-4107-afb4-75ad156ee481 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.182243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6803b-d114-4107-afb4-75ad156ee481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:58.209441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f5c6803b-d114-4107-afb4-75ad156ee481 None None] MFA Rules not processed for user `3f3e956f03cd457c91c0bfadd517dbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:58.210620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f5c6803b-d114-4107-afb4-75ad156ee481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:58.227765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:58.279137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1172/2348] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:02:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:02:58.290748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:58.292705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:58.310718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:02:58.377445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceda8b3-0dd3-43c8-bf47-795e59c58242 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:58.381986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:58.382831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.383116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.383392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:58.397114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:58.398592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:58.426568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1177/2349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:02:58.945505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5142b6-1e4c-4787-98ef-83db73e8f6c8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:58.946086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5142b6-1e4c-4787-98ef-83db73e8f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:58.946086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5142b6-1e4c-4787-98ef-83db73e8f6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:59.001927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-df5142b6-1e4c-4787-98ef-83db73e8f6c8 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:02:59.001927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-df5142b6-1e4c-4787-98ef-83db73e8f6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:59.129235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1173/2350] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 10:02:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 30 10:02:59.145312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:02:59.148372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:02:59.248337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2334bb2-51dc-436e-9c9b-809570dfebb7 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:02:59.253862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:02:59.254518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:02:59.254817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:02:59.255097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:02:59.264404 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:02:59.266006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:02:59.274826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1178/2351] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:02:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:03:01.005167 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deee92-0109-4744-a487-387ce85d4f6f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:01.006537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deee92-0109-4744-a487-387ce85d4f6f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:01.007083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deee92-0109-4744-a487-387ce85d4f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:01.029142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-49deee92-0109-4744-a487-387ce85d4f6f None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:01.030514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-49deee92-0109-4744-a487-387ce85d4f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:01.039623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1174/2352] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:03:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:01.053245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:01.064708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73108e-a1cc-45a5-b7f1-910b478edc34 service swift] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:01.073637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012a703-e25e-4b76-884a-e5cafc988566 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:01.074434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012a703-e25e-4b76-884a-e5cafc988566 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:01.075018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012a703-e25e-4b76-884a-e5cafc988566 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:01.075495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012a703-e25e-4b76-884a-e5cafc988566 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:01.084491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012a703-e25e-4b76-884a-e5cafc988566 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:01.086205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012a703-e25e-4b76-884a-e5cafc988566 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:01.095059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1179/2353] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:03:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:03:01.602277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7de798-0e8c-4c8a-a29b-e283bfbaaa9d None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:01.602564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7de798-0e8c-4c8a-a29b-e283bfbaaa9d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:01.603017 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7de798-0e8c-4c8a-a29b-e283bfbaaa9d None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:01.604624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1175/2354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:03:01.621482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:01.621772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:01.622057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:01.654341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] MFA Rules not processed for user `8641d4ad6c8a45f5b826bb43a78ee3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:01.655578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:01.780193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7de798-0e8c-4c8a-a29b-e283bfbaaa9d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:01.782616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cd7de798-0e8c-4c8a-a29b-e283bfbaaa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:01.782773 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1180/2355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:03:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:01.805621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:01.807133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:01.831955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e7da3-8471-4c18-bfcc-e37d2215c189 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:01.844684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:01.845482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:01.845833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:01.846287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:01.859449 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:01.861296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:01.875186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1181/2356] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:01.891521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7de798-0e8c-4c8a-a29b-e283bfbaaa9d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:01.905515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:01.906538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:01.907384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:01.907673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] PATH_INFO: `/v3/users/64fe0b9043ad479284163d04a90b842f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:01.911505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64fe0b9043ad479284163d04a90b842f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:01.913841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:02.005010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64fe0b9043ad479284163d04a90b842f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.006005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64fe0b9043ad479284163d04a90b842f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.016071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64fe0b9043ad479284163d04a90b842f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.016713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64fe0b9043ad479284163d04a90b842f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.023523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:02.034851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-14b94de1-62e7-46e0-92b9-2172b830e5d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64fe0b9043ad479284163d04a90b842f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.061197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1176/2357] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:03:01 2022] DELETE /identity/v3/users/64fe0b9043ad479284163d04a90b842f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:03:02.461481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:02.474030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1095084-d839-4a76-bfa0-aa078ea878fd service placement] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:02.544636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:02.657700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:02.658595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:02.658871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:02.659147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] PATH_INFO: `/v3/projects/46ae600a17f9437d9e62475696c106ca` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:02.661123 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ae600a17f9437d9e62475696c106ca)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:02.664387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:02.794024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ae600a17f9437d9e62475696c106ca', 'request_id': 'req-402cb08a-5f8e-4ba4-b9ad-3f897e063504'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.823857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ae600a17f9437d9e62475696c106ca', 'request_id': 'req-402cb08a-5f8e-4ba4-b9ad-3f897e063504'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.838354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ae600a17f9437d9e62475696c106ca', 'request_id': 'req-402cb08a-5f8e-4ba4-b9ad-3f897e063504'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.856873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] The token cache is being invalidate because project 46ae600a17f9437d9e62475696c106ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:03:02.858031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.859155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-402cb08a-5f8e-4ba4-b9ad-3f897e063504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:02.862251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1182/2358] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:03:02 2022] DELETE /identity/v3/projects/46ae600a17f9437d9e62475696c106ca => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:03:02.867707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:02.868290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:02.868573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:02.894165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:02.895795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:03.055251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1177/2359] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:03.056575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.056863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.057140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.120455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:03.123663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:03.153435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:03.169590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1183/2360] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:03:03.179090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375761-9209-47df-940f-d70f82ff1b6e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:03.188886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:03.189811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.190239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.190882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.191830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:03:03.192368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:03.194914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:03.220663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1178/2361] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:03:03.224859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:03.238231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45af9-d9e3-4280-890b-239ddb397986 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:03.243978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:03.244874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.245248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.246044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.246847 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:03.249141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:03.329011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1184/2362] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:03:03 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:03:03.329363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:03.354869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaac7-1b36-45ea-8f34-7d11d4244ead admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:03.502293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:03.503076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.503411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.503689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.504425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:03.507854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:03.565197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1179/2363] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:03:03 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:03:03.566585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:03.592998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bab848-efa5-435d-a012-e3ac15011b7c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:03.603676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:03.604644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.605156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.605576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.606616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:03.608882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:03.632218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1185/2364] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:03:03.633256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:03.649196 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190cf6c-eddd-48f1-b5f4-38b34c3a86aa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:03.668416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6162deb-aca6-47ed-b0bc-711bc208d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:03.671777 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6162deb-aca6-47ed-b0bc-711bc208d4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.672278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6162deb-aca6-47ed-b0bc-711bc208d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.675238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6162deb-aca6-47ed-b0bc-711bc208d4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.677334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6162deb-aca6-47ed-b0bc-711bc208d4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:03.685259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6162deb-aca6-47ed-b0bc-711bc208d4ea admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:03.716614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1180/2365] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:03:03.718088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:03.742017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2f501-c14b-4f93-9398-a3551c6d3d83 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:03.746904 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:03.747842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.748151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.748605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] PATH_INFO: `/v3/projects/218ddbe204144c41846be51ca0fcf612/users/615131f7497445729e12a3e83b639fde/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.765372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=218ddbe204144c41846be51ca0fcf612, user_id=615131f7497445729e12a3e83b639fde, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:03.769936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:03.818844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1186/2366] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:03:03 2022] PUT /identity/v3/projects/218ddbe204144c41846be51ca0fcf612/users/615131f7497445729e12a3e83b639fde/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:03:03.898105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885a38b-a29c-4c9e-94c1-9ddeaef8653e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:03.898396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885a38b-a29c-4c9e-94c1-9ddeaef8653e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:03.898732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885a38b-a29c-4c9e-94c1-9ddeaef8653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:03.958556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3885a38b-a29c-4c9e-94c1-9ddeaef8653e None None] MFA Rules not processed for user `615131f7497445729e12a3e83b639fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:03.961372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3885a38b-a29c-4c9e-94c1-9ddeaef8653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:04.049251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:04.100468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1181/2367] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:04.111933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:04.113714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:04.181730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:04.199933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aad446-7084-4617-b417-e023696be3d0 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:04.205595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:04.206391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:04.206752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:04.207090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:04.221799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:04.223545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:04.240984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1187/2368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:07.244119 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc3cde-462a-47cc-ab77-d3c8fa2817db None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:07.245629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc3cde-462a-47cc-ab77-d3c8fa2817db None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:07.246681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc3cde-462a-47cc-ab77-d3c8fa2817db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:07.311078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8bdc3cde-462a-47cc-ab77-d3c8fa2817db None None] MFA Rules not processed for user `a179e34636ed4970be4f0790115f9d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:07.312843 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8bdc3cde-462a-47cc-ab77-d3c8fa2817db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:07.397916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1182/2369] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:14.236130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=7 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:15.187062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:15.254306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd277-76c0-41ff-9b4c-0bd779215c1e service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:15.259499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:15.260483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:15.260848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:15.261185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] PATH_INFO: `/v3/users/615131f7497445729e12a3e83b639fde` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:15.263678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=615131f7497445729e12a3e83b639fde)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:15.264469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:15.337535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615131f7497445729e12a3e83b639fde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:15.338377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615131f7497445729e12a3e83b639fde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:15.339283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615131f7497445729e12a3e83b639fde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:15.359140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615131f7497445729e12a3e83b639fde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:15.384498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615131f7497445729e12a3e83b639fde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:15.404130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1188/2370] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:03:15 2022] DELETE /identity/v3/users/615131f7497445729e12a3e83b639fde => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:03:15.405968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:15.406466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:15.406667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:15.428697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:15.430549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:15.590963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1183/2371] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:15.601032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:15.602431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:15.694941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db27877-2151-44f4-95fd-70e83701ae9c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:15.699317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0f766-6cd3-4d68-9425-042bcbdfc882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:15.700091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0f766-6cd3-4d68-9425-042bcbdfc882 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:15.700439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0f766-6cd3-4d68-9425-042bcbdfc882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:15.700783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0f766-6cd3-4d68-9425-042bcbdfc882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:15.711821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0f766-6cd3-4d68-9425-042bcbdfc882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:15.713312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0f766-6cd3-4d68-9425-042bcbdfc882 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:15.723185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1189/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:16.234304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:16.236155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:16.249894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662924d0-e653-4881-9445-6b5c22f9d040 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:16.256533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:16.257926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:16.258219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:16.258509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] PATH_INFO: `/v3/projects/218ddbe204144c41846be51ca0fcf612` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:16.259898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] RBAC: Authorizing `identity:delete_project(project_id=218ddbe204144c41846be51ca0fcf612)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:16.261862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:16.338090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218ddbe204144c41846be51ca0fcf612', 'request_id': 'req-fb21e370-7b1a-4352-939f-b892199ca85c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:16.344562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218ddbe204144c41846be51ca0fcf612', 'request_id': 'req-fb21e370-7b1a-4352-939f-b892199ca85c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:16.352541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218ddbe204144c41846be51ca0fcf612', 'request_id': 'req-fb21e370-7b1a-4352-939f-b892199ca85c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:16.364905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:16.370037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] The token cache is being invalidate because project 218ddbe204144c41846be51ca0fcf612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:03:16.370411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:16.371452 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-fb21e370-7b1a-4352-939f-b892199ca85c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:16.374753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1184/2373] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:03:16 2022] DELETE /identity/v3/projects/218ddbe204144c41846be51ca0fcf612 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:03:21.989877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e227fb-760f-4012-84aa-aa378229eaca None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:21.990500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e227fb-760f-4012-84aa-aa378229eaca None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:21.990500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e227fb-760f-4012-84aa-aa378229eaca None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:21.991914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1190/2374] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:03:21.998290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:22.000491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:22.000952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:22.080952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:22.082082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:22.170984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1185/2375] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:03:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:24.936222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e227fb-760f-4012-84aa-aa378229eaca None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:24.937996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-09e227fb-760f-4012-84aa-aa378229eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:24.987126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:25.039337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e227fb-760f-4012-84aa-aa378229eaca None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:25.048994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:25.049676 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:25.049958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:25.050232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:25.052307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:25.141792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:25.144347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:25.156806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1191/2376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:30.443304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:30.445276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:30.557163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba7b7-9ef7-4d4e-a427-a263e964efe5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:30.562208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:30.563129 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:30.563412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:30.563691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:30.565588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:30.579116 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:30.581287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30132c72-965f-499f-b5ae-a8eee4294840 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:30.595016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1186/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:35.152620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=40 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:35.994348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:35.997232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:36.081877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a90b0c-754b-47a3-ad0f-8d25b945e725 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:36.087427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553602c1-9ea0-4f4d-bf68-e9e986094b3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:36.089466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553602c1-9ea0-4f4d-bf68-e9e986094b3c service glance] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.089876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553602c1-9ea0-4f4d-bf68-e9e986094b3c service glance] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.090141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553602c1-9ea0-4f4d-bf68-e9e986094b3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.102303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553602c1-9ea0-4f4d-bf68-e9e986094b3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:36.103741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553602c1-9ea0-4f4d-bf68-e9e986094b3c service glance] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:36.118386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1192/2378] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:36.204374 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853c507-4804-441f-8958-eca9854616df None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.205024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853c507-4804-441f-8958-eca9854616df None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.205369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853c507-4804-441f-8958-eca9854616df None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.207954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1187/2379] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:03:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:03:36.216871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c8dbc-b3dd-4fd0-b588-e1fec6f460f0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.217448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c8dbc-b3dd-4fd0-b588-e1fec6f460f0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.217448 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c8dbc-b3dd-4fd0-b588-e1fec6f460f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.240089 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-401c8dbc-b3dd-4fd0-b588-e1fec6f460f0 None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:36.241517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-401c8dbc-b3dd-4fd0-b588-e1fec6f460f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:36.252167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1193/2380] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:36.266507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853c507-4804-441f-8958-eca9854616df None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:36.275070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:36.281209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853c507-4804-441f-8958-eca9854616df None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:36.287628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a8357-4994-4937-841f-36da3f42647c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:36.288309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a8357-4994-4937-841f-36da3f42647c service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.288712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a8357-4994-4937-841f-36da3f42647c service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.288956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a8357-4994-4937-841f-36da3f42647c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.299350 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a8357-4994-4937-841f-36da3f42647c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:36.301439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a8357-4994-4937-841f-36da3f42647c service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:36.312635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1188/2381] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:36.406332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527408-1e3f-499a-a645-5990a812ff2f None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.406610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527408-1e3f-499a-a645-5990a812ff2f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.406995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527408-1e3f-499a-a645-5990a812ff2f None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.408386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1194/2382] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:03:36.414841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54259331-c2f0-4ab4-ac97-6e83643dce15 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.415171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54259331-c2f0-4ab4-ac97-6e83643dce15 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.415559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54259331-c2f0-4ab4-ac97-6e83643dce15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.442402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-54259331-c2f0-4ab4-ac97-6e83643dce15 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:36.444868 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-54259331-c2f0-4ab4-ac97-6e83643dce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:36.456724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1189/2383] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:36.469318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0527408-1e3f-499a-a645-5990a812ff2f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:36.473508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b0527408-1e3f-499a-a645-5990a812ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:36.493042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0527408-1e3f-499a-a645-5990a812ff2f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:36.497539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:36.498199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:36.498475 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:36.498730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:36.514749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:36.516815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:36.532765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1195/2384] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:39.904273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:39.904745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:39.904840 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:39.923780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:39.925002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:39.980355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1190/2385] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:03:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:40.002733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:40.004272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:40.082386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d41d1-ace6-459b-be94-606303c8a9d1 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:40.089113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09a442-ebf8-47ff-ae40-69282d8f5358 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:40.089761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09a442-ebf8-47ff-ae40-69282d8f5358 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:40.090044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09a442-ebf8-47ff-ae40-69282d8f5358 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:40.090315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09a442-ebf8-47ff-ae40-69282d8f5358 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:40.101821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09a442-ebf8-47ff-ae40-69282d8f5358 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:40.103437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09a442-ebf8-47ff-ae40-69282d8f5358 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:40.111551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1196/2386] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:03:49.884360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:49.886389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:49.966954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20e7d3-83e4-40a9-8f5f-38b693c71085 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:49.971507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:49.972433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:49.972713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:49.972988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] PATH_INFO: `/v3/users/3f3e956f03cd457c91c0bfadd517dbaa` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:49.974223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3e956f03cd457c91c0bfadd517dbaa)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:49.976341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:50.030702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3e956f03cd457c91c0bfadd517dbaa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.035953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3e956f03cd457c91c0bfadd517dbaa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.045599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3e956f03cd457c91c0bfadd517dbaa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.045969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3e956f03cd457c91c0bfadd517dbaa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.064179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3e956f03cd457c91c0bfadd517dbaa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.083003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1191/2387] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:03:49 2022] DELETE /identity/v3/users/3f3e956f03cd457c91c0bfadd517dbaa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:03:50.085007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.086155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.086921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.105463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:50.106693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:50.185476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1197/2388] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:50.192792 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:50.194408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:50.214002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:50.265702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae16e6-1f0b-4762-93ee-08e5f31ee699 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:50.269307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa3f76-113d-4b81-a4c0-7ae6f77d1428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:50.270087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa3f76-113d-4b81-a4c0-7ae6f77d1428 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.270370 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa3f76-113d-4b81-a4c0-7ae6f77d1428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.270916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa3f76-113d-4b81-a4c0-7ae6f77d1428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.284286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa3f76-113d-4b81-a4c0-7ae6f77d1428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:50.285857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa3f76-113d-4b81-a4c0-7ae6f77d1428 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:50.296663 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1192/2389] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:03:50.631693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:50.633917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:50.647429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f57a2-1c2f-41b8-8f70-9a401cb7b699 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:50.667164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:50.668111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.668392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.668670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] PATH_INFO: `/v3/projects/14cadb2391354f1f8b66756c7f087d60` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.669945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14cadb2391354f1f8b66756c7f087d60)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:50.671975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:50.683335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:50.728042 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14cadb2391354f1f8b66756c7f087d60', 'request_id': 'req-47e42366-3a85-4f0f-ae6f-8a32b1afa656'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.734068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14cadb2391354f1f8b66756c7f087d60', 'request_id': 'req-47e42366-3a85-4f0f-ae6f-8a32b1afa656'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.743815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14cadb2391354f1f8b66756c7f087d60', 'request_id': 'req-47e42366-3a85-4f0f-ae6f-8a32b1afa656'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.756583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] The token cache is being invalidate because project 14cadb2391354f1f8b66756c7f087d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:03:50.756943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.757734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-47e42366-3a85-4f0f-ae6f-8a32b1afa656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:03:50.759187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1198/2390] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:03:50 2022] DELETE /identity/v3/projects/14cadb2391354f1f8b66756c7f087d60 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:03:50.764489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.764991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.765274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.785237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:50.786460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:50.850749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1193/2391] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:03:50.859970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.860169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.860474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.881411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:50.883265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:50.891914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1199/2392] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:03:50.896247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:50.906944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753e26-47d9-48ec-a2c9-b52e08a7a23d None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:50.911507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:50.912221 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.912506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.912782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.913606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:03:50.913930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:50.915657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:50.928233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1194/2393] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:03:50.933510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:50.943223 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409f5b8-5b0c-4691-a400-a63325a11d9c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:50.948276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:50.948995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:50.949275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:50.949545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:50.950271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:50.952222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:50.990276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1200/2394] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:03:50 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:03:50.994430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:51.005919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ebc3-3426-485b-adab-7697e6e7221a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:51.066911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:51.067626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:51.067938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:51.068214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:51.069053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:51.071233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:51.096672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1195/2395] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:03:50 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:03:51.098006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:51.108351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794ec69-a767-4cb6-87b6-bae7a364593e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:51.113266 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:51.114452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:51.114918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:51.115434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:51.116495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:51.118914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:51.131030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1201/2396] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:03:51.135290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:51.145883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f44a8-f830-410d-90c1-3876989b1e74 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:51.150964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19be10-5067-4f2c-b3ad-3983eaf98fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:51.151631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19be10-5067-4f2c-b3ad-3983eaf98fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:51.151934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19be10-5067-4f2c-b3ad-3983eaf98fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:51.152210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19be10-5067-4f2c-b3ad-3983eaf98fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:51.152957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19be10-5067-4f2c-b3ad-3983eaf98fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:51.154554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19be10-5067-4f2c-b3ad-3983eaf98fc8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:51.165978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1196/2397] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:03:51.170264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:51.182839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308152b4-8184-4a87-898e-672103b8cf7b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:51.188732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:51.189642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:51.189919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:51.190189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] PATH_INFO: `/v3/projects/0af25b68baf0477c8a0d5a942dd35492/users/61d9160ee5444cde82939e9604f5f0eb/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:51.206335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af25b68baf0477c8a0d5a942dd35492, user_id=61d9160ee5444cde82939e9604f5f0eb, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:51.209023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:51.225558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1202/2398] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:03:51 2022] PUT /identity/v3/projects/0af25b68baf0477c8a0d5a942dd35492/users/61d9160ee5444cde82939e9604f5f0eb/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:03:51.236911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:51.237206 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:51.237511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:51.267972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] MFA Rules not processed for user `61d9160ee5444cde82939e9604f5f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:03:51.269338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:51.279971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:03:51.348479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1197/2399] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:03:51.359424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:03:51.361253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:03:51.442455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1f3b6-9261-4fc6-8ddc-21e4ad143d08 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:03:51.446108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faec624-0d7d-4341-8ff5-e28fdcf86c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:03:51.446780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faec624-0d7d-4341-8ff5-e28fdcf86c6f service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:03:51.447055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faec624-0d7d-4341-8ff5-e28fdcf86c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:03:51.447327 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faec624-0d7d-4341-8ff5-e28fdcf86c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:03:51.456291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faec624-0d7d-4341-8ff5-e28fdcf86c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:03:51.457726 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faec624-0d7d-4341-8ff5-e28fdcf86c6f service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:03:51.466541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1203/2400] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:03:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:04:01.342558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=24 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:01.462536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:15.028336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:04:15.099318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dcb7b-0fc2-4400-981e-a19ccb0844d0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:04:15.105281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:04:15.106239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:15.106501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:15.106766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] PATH_INFO: `/v3/users/61d9160ee5444cde82939e9604f5f0eb` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:15.108315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d9160ee5444cde82939e9604f5f0eb)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:04:15.110511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:04:15.159446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d9160ee5444cde82939e9604f5f0eb'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.160120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d9160ee5444cde82939e9604f5f0eb'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.168712 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d9160ee5444cde82939e9604f5f0eb'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.169376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d9160ee5444cde82939e9604f5f0eb'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.181799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d9160ee5444cde82939e9604f5f0eb'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.195272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1198/2401] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:04:15 2022] DELETE /identity/v3/users/61d9160ee5444cde82939e9604f5f0eb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:04:15.200126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:15.200633 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:15.200764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:15.218807 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:04:15.219921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:15.284445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1204/2402] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:04:15.294362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:04:15.295836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:15.354087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8152c-8f11-4b4a-abe4-419ed14e28cb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:04:15.358244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f84f9-103a-4c04-9758-1e241839bfc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:04:15.358884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f84f9-103a-4c04-9758-1e241839bfc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:15.359166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f84f9-103a-4c04-9758-1e241839bfc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:15.359438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f84f9-103a-4c04-9758-1e241839bfc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:15.368895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f84f9-103a-4c04-9758-1e241839bfc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:04:15.370671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f84f9-103a-4c04-9758-1e241839bfc4 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:04:15.380505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1199/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:04:15.662016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:04:15.667706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:15.682815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed0fb2-e49a-4c0b-b638-8eca0434bd35 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:04:15.687898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:04:15.688874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:15.689207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:15.689630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] PATH_INFO: `/v3/projects/0af25b68baf0477c8a0d5a942dd35492` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:15.691923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af25b68baf0477c8a0d5a942dd35492)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:04:15.693252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:04:15.745392 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af25b68baf0477c8a0d5a942dd35492', 'request_id': 'req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.751815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af25b68baf0477c8a0d5a942dd35492', 'request_id': 'req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.761280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af25b68baf0477c8a0d5a942dd35492', 'request_id': 'req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.772059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] The token cache is being invalidate because project 0af25b68baf0477c8a0d5a942dd35492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:04:15.772347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.772958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:04:15.774521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1205/2404] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:04:15 2022] DELETE /identity/v3/projects/0af25b68baf0477c8a0d5a942dd35492 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:04:17.489894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1d432-ae4d-471d-b28e-275aa6d13e8a None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:17.490388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1d432-ae4d-471d-b28e-275aa6d13e8a None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:17.490513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1d432-ae4d-471d-b28e-275aa6d13e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:17.525504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:17.547301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d6b1d432-ae4d-471d-b28e-275aa6d13e8a None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:04:17.548460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d6b1d432-ae4d-471d-b28e-275aa6d13e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:17.613862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1200/2405] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 10:04:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:04:17.627174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:04:17.628715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:17.725344 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9de49-eed2-4ca3-bc32-f823f47ec1c3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:04:17.731122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dc1ab-6cc4-4fba-ad6b-14548f6097e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:04:17.731744 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc1ab-6cc4-4fba-ad6b-14548f6097e4 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:17.732056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc1ab-6cc4-4fba-ad6b-14548f6097e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:17.732325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc1ab-6cc4-4fba-ad6b-14548f6097e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:17.737883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:17.742368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dc1ab-6cc4-4fba-ad6b-14548f6097e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:04:17.744442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dc1ab-6cc4-4fba-ad6b-14548f6097e4 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:04:17.760107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1206/2406] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:04:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:04:27.610860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=30 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:27.758872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:35.770464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4809987-781c-4759-83bc-6b3cc11936b5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:35.771004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4809987-781c-4759-83bc-6b3cc11936b5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:35.771004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4809987-781c-4759-83bc-6b3cc11936b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:35.792839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a4809987-781c-4759-83bc-6b3cc11936b5 None None] MFA Rules not processed for user `957f396e32bc48c0866a53122122bc7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:04:35.794624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a4809987-781c-4759-83bc-6b3cc11936b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:35.860047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1201/2407] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:04:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:04:45.850060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=25 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:04:54.927930 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34140d-464e-466f-adae-2ce1d6ec6a69 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:04:54.928496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34140d-464e-466f-adae-2ce1d6ec6a69 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:04:54.928496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34140d-464e-466f-adae-2ce1d6ec6a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:04:54.946194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3c34140d-464e-466f-adae-2ce1d6ec6a69 None None] MFA Rules not processed for user `4e9ad2c4909c49a6b3df8ece333e70b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:04:54.947415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3c34140d-464e-466f-adae-2ce1d6ec6a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:04:55.001867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1207/2408] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:04:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:04.994724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=25 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:15.586045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176733c8-3655-456b-aa75-5eca90660784 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:15.586867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176733c8-3655-456b-aa75-5eca90660784 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:15.586867 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176733c8-3655-456b-aa75-5eca90660784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:15.607496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-176733c8-3655-456b-aa75-5eca90660784 None None] MFA Rules not processed for user `4b1614d524d24ce486741b599268995a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:15.608729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-176733c8-3655-456b-aa75-5eca90660784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:15.676040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1202/2409] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:24.244267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:24.244862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:24.244862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:24.264354 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] MFA Rules not processed for user `a179e34636ed4970be4f0790115f9d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:24.265735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:24.330627 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1208/2410] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:24.386761 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176733c8-3655-456b-aa75-5eca90660784 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:24.388299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-176733c8-3655-456b-aa75-5eca90660784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:24.448198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176733c8-3655-456b-aa75-5eca90660784 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:24.452353 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:24.453149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:24.453518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:24.453795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:24.455595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:24.534435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:24.536097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:24.548585 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1203/2411] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:24.768743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:24.770405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:24.838960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f34acc-6857-4a2d-948a-37e8847247b3 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:24.842584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:24.843305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:24.843658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:24.844011 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:24.846185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:24.863528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:24.865230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:24.879399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1209/2412] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:34.543259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=79 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:34.872693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=51 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:40.024076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.025661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:40.092127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8438d9-3574-47d8-a901-9e97a41ac2f8 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.097324 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.098193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.098479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.098751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] PATH_INFO: `/v3/users/957f396e32bc48c0866a53122122bc7a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.100004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=957f396e32bc48c0866a53122122bc7a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.101953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:40.149645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957f396e32bc48c0866a53122122bc7a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.150322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957f396e32bc48c0866a53122122bc7a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.159492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957f396e32bc48c0866a53122122bc7a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.160125 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957f396e32bc48c0866a53122122bc7a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.172582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77272f5b-5721-445a-be1c-99dd0cefd398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957f396e32bc48c0866a53122122bc7a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.185503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1204/2413] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:05:40 2022] DELETE /identity/v3/users/957f396e32bc48c0866a53122122bc7a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:05:40.372678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.384261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc25f6f-bc1c-486d-9a26-1996b079190d service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.449283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.450147 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.450429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.450699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] PATH_INFO: `/v3/projects/8de5ed099cad4e6aa38f12d16cfb9615` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.451959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de5ed099cad4e6aa38f12d16cfb9615)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.453883 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:40.506137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de5ed099cad4e6aa38f12d16cfb9615', 'request_id': 'req-7c49b540-57cb-417e-b874-b59fd1dd41e7'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.513335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de5ed099cad4e6aa38f12d16cfb9615', 'request_id': 'req-7c49b540-57cb-417e-b874-b59fd1dd41e7'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.522047 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de5ed099cad4e6aa38f12d16cfb9615', 'request_id': 'req-7c49b540-57cb-417e-b874-b59fd1dd41e7'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.533175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] The token cache is being invalidate because project 8de5ed099cad4e6aa38f12d16cfb9615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:05:40.533441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.534092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-7c49b540-57cb-417e-b874-b59fd1dd41e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:05:40.535700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1210/2414] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:05:40 2022] DELETE /identity/v3/projects/8de5ed099cad4e6aa38f12d16cfb9615 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:05:40.541697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.542098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.542454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.564345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:40.565467 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:40.633012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1205/2415] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:40.696390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.696663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.696941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.719135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:40.720465 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:40.731907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1211/2416] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:40.736856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.747235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175003-b3c9-423c-897d-a43cb98afabd None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.753733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.754394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.754674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.754943 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.755681 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:05:40.756026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.757646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:40.770661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1206/2417] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:05:40.776836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.787515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cf712-d36f-445d-8761-5136ec3d56ef None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.793912 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.794565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.794851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.795129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.795824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.797782 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:40.826550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1212/2418] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:05:40 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:05:40.831283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.846558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef338-7975-4fff-866c-1e3a69948112 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.850519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:40.905852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.906523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.906807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.907085 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.907913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.909873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:40.933515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1207/2419] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:05:40 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:05:40.936733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.948409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce5e4d-539d-4cea-b172-0feb55f317fc admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.954634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.955289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.955573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.955805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.956584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.958745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:40.969112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1213/2420] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:05:40.973352 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:40.983247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127ffb-008a-408b-a11e-032614ff376c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:40.988248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7bda5-fc4b-45f5-b8c4-a13e96ec4055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:40.988898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7bda5-fc4b-45f5-b8c4-a13e96ec4055 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:40.989179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7bda5-fc4b-45f5-b8c4-a13e96ec4055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:40.989449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7bda5-fc4b-45f5-b8c4-a13e96ec4055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:40.990158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7bda5-fc4b-45f5-b8c4-a13e96ec4055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:40.991752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7bda5-fc4b-45f5-b8c4-a13e96ec4055 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:41.001984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1208/2421] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:05:41.006424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:41.016792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295b2c-d96f-4464-8eda-47071c7bb922 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:41.022625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:41.023518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:41.023798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:41.024092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] PATH_INFO: `/v3/projects/108ee256fba04d1b83fe00c166742f0b/users/ead8ee09f8514e75882c877042764a92/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:41.041334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] RBAC: Authorizing `identity:create_grant(project_id=108ee256fba04d1b83fe00c166742f0b, user_id=ead8ee09f8514e75882c877042764a92, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:41.046326 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:41.065552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1214/2422] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:05:41 2022] PUT /identity/v3/projects/108ee256fba04d1b83fe00c166742f0b/users/ead8ee09f8514e75882c877042764a92/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:05:41.073140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0653ccd-0f38-440f-966c-e2a9fe1bfa03 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:41.073640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0653ccd-0f38-440f-966c-e2a9fe1bfa03 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:41.073758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0653ccd-0f38-440f-966c-e2a9fe1bfa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:41.099511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f0653ccd-0f38-440f-966c-e2a9fe1bfa03 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:41.100746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f0653ccd-0f38-440f-966c-e2a9fe1bfa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:41.161313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1209/2423] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:41.171093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:41.172978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:41.207572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:41.239660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bf917-7c13-4f98-86d3-89d63b2e00de admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:41.243902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:41.244558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:41.244833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:41.245099 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:41.255022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:41.256500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:41.266373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1215/2424] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:45.130479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.131005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.131005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.161502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] MFA Rules not processed for user `ead8ee09f8514e75882c877042764a92`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:45.162823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:45.237334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1210/2425] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:45.241774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:45.251806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6296e2-6170-45b7-8448-30e0976637bd service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:45.256391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:45.257052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.257332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.257606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.258308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:45.260236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:45.291493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1216/2426] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:05:45 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:05:45.295694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:45.307662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45aae4-b6d0-4db9-8533-20893ea4d7f6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:45.321157 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:45.368954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:45.369626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.369919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.370214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.370906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:45.372918 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:45.395972 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1211/2427] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:05:45 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:05:45.400110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:45.412503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae38ef-6c48-4cca-ba8f-e785d8b8aece admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:45.417451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:45.418229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.418576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.418922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.419764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:45.421479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:45.436287 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1217/2428] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:05:45.442268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:45.456392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497e117-a47c-4c05-b8b6-6ea475649bf8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:45.464514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:45.465856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.466244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.466638 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] PATH_INFO: `/v3/projects/f6bdfc275275414fa807b014ea3360cc/users/504258e47adc476d92eb3db56e71126d/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.483538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdfc275275414fa807b014ea3360cc, user_id=504258e47adc476d92eb3db56e71126d, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:45.486509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:45.504734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1212/2429] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:05:45 2022] PUT /identity/v3/projects/f6bdfc275275414fa807b014ea3360cc/users/504258e47adc476d92eb3db56e71126d/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:05:45.508997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:45.519761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc418e0-7050-4469-ba1d-e6708e0d0d17 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:45.592503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:45.593395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.593788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.594096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.595215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:45.596897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:45.607812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1218/2430] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:05:45.612009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:45.623698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7e2a5-63e9-412c-89fe-6c382b66d0ff admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:45.629968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:45.630866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:45.631148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:45.631422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] PATH_INFO: `/v3/projects/f6bdfc275275414fa807b014ea3360cc/users/504258e47adc476d92eb3db56e71126d/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:45.633489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdfc275275414fa807b014ea3360cc, user_id=504258e47adc476d92eb3db56e71126d, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:45.636231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:45.646916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1213/2431] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:05:45 2022] PUT /identity/v3/projects/f6bdfc275275414fa807b014ea3360cc/users/504258e47adc476d92eb3db56e71126d/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:05:46.569701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:46.683963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4abb1-b8c5-436b-8ec7-6287faa9a415 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:46.769671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:46.777271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:46.778751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:46.779238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:46.779692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:46.782533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:46.798182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:46.800634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099aa4c-748a-47ed-8c29-5f9f5aa64ba9 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:46.815183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1219/2432] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:49.452025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:49.454687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:49.569629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b736ff-ce58-40fb-8ff8-b8c6091a84fa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:49.575166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:49.577291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:49.577291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:49.577291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:49.580521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:49.687752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:49.689919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:49.721203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1214/2433] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:50.734217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:50.734820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:50.734820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:50.764477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] MFA Rules not processed for user `504258e47adc476d92eb3db56e71126d`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:50.765625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:50.840428 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1220/2434] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:05:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:50.855135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:50.862722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:05:50.866832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fb6ac-0831-40f9-b606-dcb2bac97e0e service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:50.873793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d43187-6e9a-4f18-931c-e20c491ec391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:50.874432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43187-6e9a-4f18-931c-e20c491ec391 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:50.874724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43187-6e9a-4f18-931c-e20c491ec391 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:50.875008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d43187-6e9a-4f18-931c-e20c491ec391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:50.934271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43187-6e9a-4f18-931c-e20c491ec391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:50.935892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d43187-6e9a-4f18-931c-e20c491ec391 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:50.946053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1215/2435] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:53.962005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:53.963762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:53.986567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44152bd4-8a4d-45af-8e55-cdc39dcec632 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:53.992989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:53.993651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:53.993932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:53.994206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:53.995989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:54.009258 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:54.010856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:54.022866 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1221/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:05:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:05:55.120554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:55.121069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:55.121069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:55.140247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:05:55.141601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:55.215429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1216/2437] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 10:05:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:05:55.267370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:05:55.268838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:05:55.347506 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea71d3-27ae-4f84-80f9-df55194a63bf service nova] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:05:55.353323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:05:55.354093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:05:55.354379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:05:55.354651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:05:55.364006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:05:55.365629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:05:55.374236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1222/2438] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:05:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:06:03.029640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:03.031627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:03.110457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90e9ed-cb97-40ce-8172-1aeea7aec6c2 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:03.115052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:03.115958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:03.116261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:03.116536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] PATH_INFO: `/v3/users/aa782d84840842ef983ed058379d5626` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:03.117747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa782d84840842ef983ed058379d5626)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:03.122012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:03.179554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa782d84840842ef983ed058379d5626'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.180649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa782d84840842ef983ed058379d5626'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.185758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:03.193511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa782d84840842ef983ed058379d5626'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.194376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa782d84840842ef983ed058379d5626'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.208443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa782d84840842ef983ed058379d5626'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.221644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1217/2439] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:06:03 2022] DELETE /identity/v3/users/aa782d84840842ef983ed058379d5626 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:03.408547 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:03.420372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92bc87-f84a-4888-87b3-99a96b7e473a service swift] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:03.480027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:03.493203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:03.494063 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:03.494339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:03.494609 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] PATH_INFO: `/v3/projects/d589375d668e413d839c63b0c3d4f104` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:03.495794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d589375d668e413d839c63b0c3d4f104)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:03.497739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:03.549006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd589375d668e413d839c63b0c3d4f104', 'request_id': 'req-86010cd4-4302-46ff-b225-55a9783078bf'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.554515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd589375d668e413d839c63b0c3d4f104', 'request_id': 'req-86010cd4-4302-46ff-b225-55a9783078bf'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.564116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd589375d668e413d839c63b0c3d4f104', 'request_id': 'req-86010cd4-4302-46ff-b225-55a9783078bf'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.576541 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] The token cache is being invalidate because project d589375d668e413d839c63b0c3d4f104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:06:03.576848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.577548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.580074 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1223/2440] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:06:03 2022] DELETE /identity/v3/projects/d589375d668e413d839c63b0c3d4f104 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:03.584104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:03.585679 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:03.599014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99822b62-7e6b-4aac-81c8-3766e6b17eb1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:03.655436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:03.656325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:03.656713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:03.656915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] PATH_INFO: `/v3/users/ccce04f4f436432bbacc0a66ba6a52a9` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:03.658619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccce04f4f436432bbacc0a66ba6a52a9)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:03.660601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:03.712625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccce04f4f436432bbacc0a66ba6a52a9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.713368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccce04f4f436432bbacc0a66ba6a52a9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.721949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccce04f4f436432bbacc0a66ba6a52a9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.722633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccce04f4f436432bbacc0a66ba6a52a9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.735551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-22eb68e6-6cdf-4b3e-81fe-7ac0c9307eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccce04f4f436432bbacc0a66ba6a52a9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:03.748828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1218/2441] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:06:03 2022] DELETE /identity/v3/users/ccce04f4f436432bbacc0a66ba6a52a9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:03.939372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:03.940966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:03.956863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86010cd4-4302-46ff-b225-55a9783078bf admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:04.016383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:04.017487 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:04.017770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:04.018046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] PATH_INFO: `/v3/projects/7fb082c5eb6f43d7b0aa99b2a003f518` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:04.019468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb082c5eb6f43d7b0aa99b2a003f518)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:04.021477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:04.076896 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb082c5eb6f43d7b0aa99b2a003f518', 'request_id': 'req-c5b24f98-ceea-4733-99c9-700836c9bd0e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:04.082986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb082c5eb6f43d7b0aa99b2a003f518', 'request_id': 'req-c5b24f98-ceea-4733-99c9-700836c9bd0e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:04.095664 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb082c5eb6f43d7b0aa99b2a003f518', 'request_id': 'req-c5b24f98-ceea-4733-99c9-700836c9bd0e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:04.107983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] The token cache is being invalidate because project 7fb082c5eb6f43d7b0aa99b2a003f518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:06:04.108271 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:04.108914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:04.111927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1224/2442] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:06:03 2022] DELETE /identity/v3/projects/7fb082c5eb6f43d7b0aa99b2a003f518 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:08.568671 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b181-ce17-4eb1-b945-034b3f00a8cd None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:08.569289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b181-ce17-4eb1-b945-034b3f00a8cd None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:08.569289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b181-ce17-4eb1-b945-034b3f00a8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:08.588880 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1360b181-ce17-4eb1-b945-034b3f00a8cd None None] MFA Rules not processed for user `4e9ad2c4909c49a6b3df8ece333e70b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:08.590072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1360b181-ce17-4eb1-b945-034b3f00a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:08.633403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:08.649298 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1219/2443] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:08.957217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:08.958790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:08.975051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:09.025747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24f98-ceea-4733-99c9-700836c9bd0e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:09.029859 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:09.030508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:09.030834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:09.031105 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:09.033078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:09.042887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:09.044362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:09.052733 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1225/2444] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:09.060920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:09.061329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:09.061612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:09.080182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] MFA Rules not processed for user `ead8ee09f8514e75882c877042764a92`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:09.081386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:09.140168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1220/2445] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:09.391050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:09.392629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:09.469821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f0ee-c77d-4716-9844-f3d61605942e service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:09.474720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:09.475367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:09.475644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:09.475941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:09.477708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:09.489300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:09.490876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:09.500784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1226/2446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:09.504051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:09.514509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851ca1b-46de-4469-b746-c960603fe1f8 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:09.518131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:09.518765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:09.519039 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:09.519312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:09.521133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:09.531087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:09.532827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:09.540713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1221/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:11.508288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:11.509924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:11.627034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1c12d-cbc1-4190-8e9d-709553d69e94 service placement] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:11.630993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:11.631658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:11.631975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:11.632251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:11.634078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:11.710835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:11.712351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:11.719039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:11.726374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1227/2448] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:19.537531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=35 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:20.794006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:20.795544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:20.871611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be301a-5dca-4a8a-92af-663c1df8c3d1 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:20.878332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:20.879299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:20.879299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:20.879524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:20.881536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:20.948900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:20.950569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:20.960342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1222/2449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:22.826834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:22.829139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:22.848670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b4d9-ea59-40f9-b81d-98c45c346e75 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:22.854739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:22.855631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:22.856035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:22.856435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:22.859735 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:22.873044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:22.874856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:22.886737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1228/2450] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:28.616220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:28.628667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c81f5-e93d-451c-8e26-2be3a9586cfb service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:28.635114 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:28.635785 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:28.636115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:28.636396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:28.639982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:28.653243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:28.654864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:28.666311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1223/2451] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:31.531748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:31.533379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:31.545018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963cbde-e6c8-4f5c-a166-116af4bf738b service nova] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:31.549104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:31.549743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:31.550018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:31.550277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:31.552203 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:31.619510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:31.621370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:31.632199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1229/2452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:31.788547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:31.799132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eab1d4-9e58-4f60-801e-85683cc6207b service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:31.803392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:31.804086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:31.804363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:31.804626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:31.806780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:31.873633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:31.875009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:31.885839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1224/2453] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:35.003471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.005102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:35.017836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fceaa6a-d250-4006-b78b-eabae7e5989c service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.026516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.027746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.028065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.028338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] PATH_INFO: `/v3/users/a179e34636ed4970be4f0790115f9d8f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.029554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a179e34636ed4970be4f0790115f9d8f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.031570 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.093337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a179e34636ed4970be4f0790115f9d8f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.094086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a179e34636ed4970be4f0790115f9d8f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.094813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a179e34636ed4970be4f0790115f9d8f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.102242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a179e34636ed4970be4f0790115f9d8f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.114970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-68d44b47-e7f3-43e5-ba42-cb781660ca64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a179e34636ed4970be4f0790115f9d8f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.127740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1230/2454] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:06:35 2022] DELETE /identity/v3/users/a179e34636ed4970be4f0790115f9d8f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:35.344582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.359758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d571-cb40-4ec6-9415-6e85c2379e15 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.374857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:35.420842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.421819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.422239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.422582 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] PATH_INFO: `/v3/projects/0e8bc4e999ce4f1e9f52b456bcdd88d3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.423996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8bc4e999ce4f1e9f52b456bcdd88d3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.425981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.474496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8bc4e999ce4f1e9f52b456bcdd88d3', 'request_id': 'req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.480076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8bc4e999ce4f1e9f52b456bcdd88d3', 'request_id': 'req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.486892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8bc4e999ce4f1e9f52b456bcdd88d3', 'request_id': 'req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.500627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] The token cache is being invalidate because project 0e8bc4e999ce4f1e9f52b456bcdd88d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:06:35.500992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.501680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3168b0d4-045c-4bdb-9008-7edcc7ea2ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:06:35.503346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1225/2455] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:06:35 2022] DELETE /identity/v3/projects/0e8bc4e999ce4f1e9f52b456bcdd88d3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:35.508397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.508705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.508978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.526851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:35.528081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:35.581976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:35.596581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1231/2456] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:35.605659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.605946 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.606281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.627248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:35.628474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:35.638058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1226/2457] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:35.643945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.658631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85ac96-d841-4dda-b34b-047de855c547 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.664781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.665439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.665717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.665986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.666712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:06:35.667015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.668665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.680988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1232/2458] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:06:35.685504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.696021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b072e-79fe-4c81-9114-14bef6b283a6 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.702230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.703130 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.703548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.703982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.704805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.706987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.735287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1227/2459] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:06:35 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:06:35.739812 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.750343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192f8db-5089-4983-a5ad-5d1e693f0924 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.813960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.814665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.814947 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.815222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.815988 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.817944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.838970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1233/2460] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:06:35 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:06:35.843348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.857678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846a54-2695-42b0-8ab6-adfc938cdf02 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.864351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.865006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.865294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.865570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.866307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.867915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.879058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1228/2461] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:06:35.883056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.893838 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4338-0368-4adc-ba17-63c79cec44fa admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.898720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6f5e4-4c71-473b-8236-f29462306661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.899382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f5e4-4c71-473b-8236-f29462306661 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.899662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f5e4-4c71-473b-8236-f29462306661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.899978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f5e4-4c71-473b-8236-f29462306661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.900706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f5e4-4c71-473b-8236-f29462306661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.902295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f5e4-4c71-473b-8236-f29462306661 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.913395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1234/2462] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:06:35.917882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:35.928787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbc999-3429-42b7-9762-507581ea4e48 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:35.934380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:35.935277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.935557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.935846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] PATH_INFO: `/v3/projects/315760a46046430780d7d4741cd8dc7f/users/3593c14c5890431d8c4a986132a659b4/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.953108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315760a46046430780d7d4741cd8dc7f, user_id=3593c14c5890431d8c4a986132a659b4, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:35.956355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:35.972595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1229/2463] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:06:35 2022] PUT /identity/v3/projects/315760a46046430780d7d4741cd8dc7f/users/3593c14c5890431d8c4a986132a659b4/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:06:35.977568 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52864e-4575-426c-9070-612717183ba9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:35.977857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52864e-4575-426c-9070-612717183ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:35.978132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52864e-4575-426c-9070-612717183ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:35.997788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-dd52864e-4575-426c-9070-612717183ba9 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:35.999009 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dd52864e-4575-426c-9070-612717183ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:36.067853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1235/2464] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:06:36.078501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:36.081468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:36.136155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:36.150894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054722-ee3d-465c-b008-176a42a8c039 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:36.154987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:36.155701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:36.156061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:36.156351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:36.168619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:36.170135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:36.184472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1230/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:36.245467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604633-00ef-4cc8-904a-246bc00670b1 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:36.245756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604633-00ef-4cc8-904a-246bc00670b1 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:36.246032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604633-00ef-4cc8-904a-246bc00670b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:36.294423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e2604633-00ef-4cc8-904a-246bc00670b1 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:36.295014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e2604633-00ef-4cc8-904a-246bc00670b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:36.369720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1236/2466] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:06:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:36.382419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:36.385328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:36.473635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e584b1-282b-4ced-9422-92d6eb69178a service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:36.480424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:36.481448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:36.481614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:36.481893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:36.492468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:36.494553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:36.504648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1231/2467] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:06:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:06:41.870584 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cab5de-c16c-446a-9011-8121687f03ff None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:41.871023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cab5de-c16c-446a-9011-8121687f03ff None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:41.871212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cab5de-c16c-446a-9011-8121687f03ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:41.890538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:41.933280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-f2cab5de-c16c-446a-9011-8121687f03ff None None] MFA Rules not processed for user `3593c14c5890431d8c4a986132a659b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:41.934936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f2cab5de-c16c-446a-9011-8121687f03ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:42.063608 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1237/2468] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:42.080616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:42.083613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:42.213534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ab7a2-263c-4212-909a-e1fe504dac5f service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:42.220115 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b58c87-6501-413c-aa33-5772405a5b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:42.220878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58c87-6501-413c-aa33-5772405a5b11 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:42.221219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58c87-6501-413c-aa33-5772405a5b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:42.221955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58c87-6501-413c-aa33-5772405a5b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:42.231168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b58c87-6501-413c-aa33-5772405a5b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:42.233318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b58c87-6501-413c-aa33-5772405a5b11 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:42.245905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1232/2469] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:44.003167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505fe20-fce2-40fc-8d64-f3067440e3da None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:44.004140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505fe20-fce2-40fc-8d64-f3067440e3da None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:44.004423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505fe20-fce2-40fc-8d64-f3067440e3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:44.050153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9505fe20-fce2-40fc-8d64-f3067440e3da None None] MFA Rules not processed for user `3593c14c5890431d8c4a986132a659b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:44.051517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9505fe20-fce2-40fc-8d64-f3067440e3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:44.063942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1238/2470] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:06:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:52.242217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=72 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:54.022624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=33 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:06:55.116844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccd8b1-d367-496a-ab2a-2ebc50f4dadb None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.117802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccd8b1-d367-496a-ab2a-2ebc50f4dadb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.118161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccd8b1-d367-496a-ab2a-2ebc50f4dadb None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.120884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1233/2471] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:06:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:06:55.129441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03c74a-a79a-4219-a840-935baabc51ec None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.130511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03c74a-a79a-4219-a840-935baabc51ec None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.130944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03c74a-a79a-4219-a840-935baabc51ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.149887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4b03c74a-a79a-4219-a840-935baabc51ec None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:55.151339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4b03c74a-a79a-4219-a840-935baabc51ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:55.162157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1239/2472] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:06:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:55.175820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccd8b1-d367-496a-ab2a-2ebc50f4dadb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:55.177739 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eeccd8b1-d367-496a-ab2a-2ebc50f4dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:55.276806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccd8b1-d367-496a-ab2a-2ebc50f4dadb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:55.281971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365e0c-cc47-4b79-ac26-188f8c3d3933 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:55.282631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365e0c-cc47-4b79-ac26-188f8c3d3933 service glance] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.283010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365e0c-cc47-4b79-ac26-188f8c3d3933 service glance] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.283231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365e0c-cc47-4b79-ac26-188f8c3d3933 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.302267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365e0c-cc47-4b79-ac26-188f8c3d3933 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:55.304289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365e0c-cc47-4b79-ac26-188f8c3d3933 service glance] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:55.318964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1234/2473] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:55.388653 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb02a-a165-4ae8-aa60-ead8f0ed9910 None None] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.388890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb02a-a165-4ae8-aa60-ead8f0ed9910 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.389167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb02a-a165-4ae8-aa60-ead8f0ed9910 None None] PATH_INFO: `/` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.390963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1240/2474] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:06:55.395927 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e1d8e-b3f5-4d3d-ab86-55c190202128 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.396244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e1d8e-b3f5-4d3d-ab86-55c190202128 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.396543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e1d8e-b3f5-4d3d-ab86-55c190202128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.415334 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a05e1d8e-b3f5-4d3d-ab86-55c190202128 None None] MFA Rules not processed for user `7b086db83a6b47fb98d450bb7e1c8026`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:55.416634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a05e1d8e-b3f5-4d3d-ab86-55c190202128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:55.424214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1235/2475] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:06:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:55.435975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb02a-a165-4ae8-aa60-ead8f0ed9910 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:55.438139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9a9fb02a-a165-4ae8-aa60-ead8f0ed9910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:55.451672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb02a-a165-4ae8-aa60-ead8f0ed9910 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:55.455381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e7528-11a2-4e8b-91ac-b2471276326e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:55.456040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e7528-11a2-4e8b-91ac-b2471276326e service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.456319 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e7528-11a2-4e8b-91ac-b2471276326e service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.456592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e7528-11a2-4e8b-91ac-b2471276326e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.464542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e7528-11a2-4e8b-91ac-b2471276326e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:55.466015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e7528-11a2-4e8b-91ac-b2471276326e service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:55.475846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1241/2476] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:55.569682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ae278-56f3-4419-9ab5-1ee351e18b43 None None] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.569974 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ae278-56f3-4419-9ab5-1ee351e18b43 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.570293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ae278-56f3-4419-9ab5-1ee351e18b43 None None] PATH_INFO: `/` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.571713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1236/2477] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 10:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:06:55.577236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa0c28-7318-422b-a21f-ce03791fa1a4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.577565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa0c28-7318-422b-a21f-ce03791fa1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.577848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa0c28-7318-422b-a21f-ce03791fa1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.599029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-dbaa0c28-7318-422b-a21f-ce03791fa1a4 None None] MFA Rules not processed for user `14dd1de1fe914f319017818e571732de`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:55.600300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dbaa0c28-7318-422b-a21f-ce03791fa1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:55.611192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1242/2478] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 10:06:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:55.622140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ae278-56f3-4419-9ab5-1ee351e18b43 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:55.634781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ae278-56f3-4419-9ab5-1ee351e18b43 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:55.638816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:55.639481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:55.639794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:55.641390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:55.652142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:55.653613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:55.665323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1237/2479] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:06:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:06:58.200729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ecc8d-7245-4949-a919-5d6c03925d8c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:58.201205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ecc8d-7245-4949-a919-5d6c03925d8c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:58.201318 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ecc8d-7245-4949-a919-5d6c03925d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:58.219059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ec6ecc8d-7245-4949-a919-5d6c03925d8c None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:06:58.220316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ec6ecc8d-7245-4949-a919-5d6c03925d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:06:58.227803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1243/2480] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:06:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:06:58.249973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:06:58.261409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638ed15-3d1a-4014-879f-6b0965f6a1d5 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:06:58.267177 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:06:58.267808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:06:58.268200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:06:58.268476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:06:58.282667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:06:58.284503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:06:58.297415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1238/2481] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:07:08.213743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=15 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:08.295423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=50 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:14.659078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:14.659606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:14.659606 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:14.679161 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:14.680396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:14.687989 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1244/2482] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 10:07:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:14.700641 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:14.711470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869886b-eb49-4cb3-9065-decf26a21fa2 service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:14.716861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:14.717501 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:14.717776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:14.718049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:14.726149 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:14.727726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:14.735855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1239/2483] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:07:24.673895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:24.732832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=6 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:26.129517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:26.138909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11719383-9378-47f2-a17e-48588f363fb7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:26.142770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:26.143410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:26.143686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:26.143991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:26.145698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:26.157780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:26.159213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:26.169519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1245/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:29.325998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:29.327730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:29.343391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5b192-9fb1-4c37-952f-310b5c14a00d service swift] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:29.348030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:29.348933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:29.349211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:29.349484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] PATH_INFO: `/v3/users/4e9ad2c4909c49a6b3df8ece333e70b5` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:29.350747 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9ad2c4909c49a6b3df8ece333e70b5)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:29.352693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:29.408504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9ad2c4909c49a6b3df8ece333e70b5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.409300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9ad2c4909c49a6b3df8ece333e70b5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.419451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9ad2c4909c49a6b3df8ece333e70b5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.420589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9ad2c4909c49a6b3df8ece333e70b5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.433105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-011d0c4c-37c9-4bd1-b018-e9ee6d56b759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9ad2c4909c49a6b3df8ece333e70b5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.446231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1240/2485] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:29 2022] DELETE /identity/v3/users/4e9ad2c4909c49a6b3df8ece333e70b5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:29.637645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:29.655508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c0068-0ad1-4978-a88b-f5cefeb978f5 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:29.724822 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:29.725594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:29.725871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:29.726140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] PATH_INFO: `/v3/projects/2e55dd4580184f158efedbad5dc6e8fb` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:29.727307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e55dd4580184f158efedbad5dc6e8fb)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:29.729292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:29.778107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e55dd4580184f158efedbad5dc6e8fb', 'request_id': 'req-ad4436f2-db29-4b75-bf56-defe3fd13bb0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.784347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e55dd4580184f158efedbad5dc6e8fb', 'request_id': 'req-ad4436f2-db29-4b75-bf56-defe3fd13bb0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.792906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e55dd4580184f158efedbad5dc6e8fb', 'request_id': 'req-ad4436f2-db29-4b75-bf56-defe3fd13bb0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.804340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] The token cache is being invalidate because project 2e55dd4580184f158efedbad5dc6e8fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:29.804622 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.805238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ad4436f2-db29-4b75-bf56-defe3fd13bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:29.806757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1246/2486] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:29 2022] DELETE /identity/v3/projects/2e55dd4580184f158efedbad5dc6e8fb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:29.812315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:29.812601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:29.812873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:29.833245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:29.834453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:29.900951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1241/2487] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:29.910597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:29.910875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:29.911144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:29.929798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:29.931007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:29.939473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1247/2488] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:29.943766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:29.953483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ba051-87a8-4cae-b0fc-57db7caeca40 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:29.958364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:29.959008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:29.959253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:29.959541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:29.960299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:07:29.960569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:29.962164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:29.974427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1242/2489] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:07:29.979417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:29.991839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d108ffc-2249-4c50-a388-dcf1066f384d None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:29.997713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:29.998381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:29.998745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:29.999250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.000038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:30.001948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:30.041817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1248/2490] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:29 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:30.043765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:30.057978 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a003a-9591-4ecb-9743-1faaf1d7d150 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:30.115088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:30.115752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:30.116078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:30.116354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.117027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:30.118958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:30.125900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:30.142715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1243/2491] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:30 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:30.147040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:30.157559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546995f2-aa37-4479-9d2b-a1eb0e7e7caa admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:30.163045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:30.163703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:30.164043 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:30.164303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.165156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:30.167056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:30.178006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1249/2492] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:30.182317 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:30.192942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b65ec-e75e-4f52-a052-35384d47ae2c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:30.198355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9bf01-504b-4669-b192-02e8edaa1a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:30.199024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9bf01-504b-4669-b192-02e8edaa1a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:30.199307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9bf01-504b-4669-b192-02e8edaa1a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:30.199583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9bf01-504b-4669-b192-02e8edaa1a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.200364 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9bf01-504b-4669-b192-02e8edaa1a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:30.202032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9bf01-504b-4669-b192-02e8edaa1a87 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:30.212942 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1244/2493] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:30.217144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:30.228040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07136de6-621e-4450-aa11-0b1df6fa7ebf admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:30.234029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:30.234920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:30.235196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:30.235469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] PATH_INFO: `/v3/projects/69ceacfd4ed140efaa0865923b747bda/users/b0b6c0d6ddd442b79d168699bcd6842e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.252800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ceacfd4ed140efaa0865923b747bda, user_id=b0b6c0d6ddd442b79d168699bcd6842e, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:30.255710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:30.271131 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1250/2494] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:30 2022] PUT /identity/v3/projects/69ceacfd4ed140efaa0865923b747bda/users/b0b6c0d6ddd442b79d168699bcd6842e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:30.276448 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612b7e4-3fe2-4801-9153-6549660d2db4 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:30.276818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612b7e4-3fe2-4801-9153-6549660d2db4 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:30.277151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612b7e4-3fe2-4801-9153-6549660d2db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.297127 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b612b7e4-3fe2-4801-9153-6549660d2db4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:30.298335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b612b7e4-3fe2-4801-9153-6549660d2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:30.361440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1245/2495] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:30.371129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:30.373024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:30.379722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:30.431245 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee3f6b-10ba-4e46-81db-d0e92c0b413e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:30.435590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:30.436720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:30.437005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:30.437279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:30.446610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:30.448078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:30.457752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1251/2496] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:31.108822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.109194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.109554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.136878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] MFA Rules not processed for user `b0b6c0d6ddd442b79d168699bcd6842e`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:31.138133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:31.206408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1246/2497] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:31.210984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:31.221216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233deb2-8724-43a7-bce6-783302027cc4 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:31.225902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:31.226572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.226853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.227126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.227834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:31.229775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:31.257797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1252/2498] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:31 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:31.262021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:31.273343 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3db9c-1cdc-40d0-86d8-9a2121798e49 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:31.337869 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:31.338543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.338822 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.339095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.339802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:31.341767 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:31.357095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:31.363698 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1247/2499] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:31 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:07:31.367818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:31.378669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a938f-d142-46a9-8fa1-385e72b1ac0c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:31.384038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:31.384699 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.384975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.385242 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.385962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:31.387572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:31.399062 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1253/2500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:31.403551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:31.414609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edb2bc-2fd4-4496-9ece-64b08c524282 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:31.420916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:31.421800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.422101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.422371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] PATH_INFO: `/v3/projects/a84c540dd163496f91bc331639ad61fa/users/67d46d5f0032460c8417f8f6daf2abbd/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.438707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84c540dd163496f91bc331639ad61fa, user_id=67d46d5f0032460c8417f8f6daf2abbd, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:31.441377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:31.457328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1248/2501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:31 2022] PUT /identity/v3/projects/a84c540dd163496f91bc331639ad61fa/users/67d46d5f0032460c8417f8f6daf2abbd/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:31.463262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:31.475618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811291d7-c64b-4f24-91ca-dcbb66000759 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:31.536698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdd26e-886b-4637-abdd-593757505a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:31.537418 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdd26e-886b-4637-abdd-593757505a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.537641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdd26e-886b-4637-abdd-593757505a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.537909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdd26e-886b-4637-abdd-593757505a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.538638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdd26e-886b-4637-abdd-593757505a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:31.540283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdd26e-886b-4637-abdd-593757505a2f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:31.550763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1254/2502] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:31.556417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:31.568773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bcbee-a4b6-4e35-ac10-8b63d98dee91 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:31.574605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:31.575496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:31.575775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:31.576078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] PATH_INFO: `/v3/projects/a84c540dd163496f91bc331639ad61fa/users/67d46d5f0032460c8417f8f6daf2abbd/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:31.578369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84c540dd163496f91bc331639ad61fa, user_id=67d46d5f0032460c8417f8f6daf2abbd, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:31.581104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:31.593592 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1249/2503] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:31 2022] PUT /identity/v3/projects/a84c540dd163496f91bc331639ad61fa/users/67d46d5f0032460c8417f8f6daf2abbd/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:32.256218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:32.256772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:32.256772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:32.285068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] MFA Rules not processed for user `67d46d5f0032460c8417f8f6daf2abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:32.286290 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:32.353409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:32.365182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1255/2504] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:32.844752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:32.846489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:32.933012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0edea-bf4e-474f-93cf-ee8919dec598 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:32.938554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:32.939210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:32.939484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:32.939754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:32.941568 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:32.953303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:32.954906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:32.965339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1250/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:33.443690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:33.507026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f1a3d-761d-4a33-b42e-a90f4baa4415 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:33.512482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:33.513638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:33.514078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:33.514479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] PATH_INFO: `/v3/users/b0b6c0d6ddd442b79d168699bcd6842e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:33.515885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b6c0d6ddd442b79d168699bcd6842e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:33.517865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:33.573023 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b6c0d6ddd442b79d168699bcd6842e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.573697 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b6c0d6ddd442b79d168699bcd6842e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.574293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b6c0d6ddd442b79d168699bcd6842e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.582574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b6c0d6ddd442b79d168699bcd6842e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.596184 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b6c0d6ddd442b79d168699bcd6842e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.609227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1256/2506] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:33 2022] DELETE /identity/v3/users/b0b6c0d6ddd442b79d168699bcd6842e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:33.787887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:33.800367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284bbdd-29a7-4e1b-b2b5-bcfd9c0b2505 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:33.861305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:33.862194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:33.862468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:33.862737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] PATH_INFO: `/v3/projects/69ceacfd4ed140efaa0865923b747bda` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:33.864025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ceacfd4ed140efaa0865923b747bda)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:33.865899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:33.914813 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ceacfd4ed140efaa0865923b747bda', 'request_id': 'req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.916741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:33.922862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ceacfd4ed140efaa0865923b747bda', 'request_id': 'req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.931539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ceacfd4ed140efaa0865923b747bda', 'request_id': 'req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.947164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] The token cache is being invalidate because project 69ceacfd4ed140efaa0865923b747bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:33.947455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.948140 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:33.949697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1251/2507] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:33 2022] DELETE /identity/v3/projects/69ceacfd4ed140efaa0865923b747bda => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:33.953700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:33.955117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:33.969324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce242392-f55f-49b2-b5ff-fadabc8a269f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.027910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.028780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.029055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.029323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] PATH_INFO: `/v3/users/67d46d5f0032460c8417f8f6daf2abbd` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.031138 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d46d5f0032460c8417f8f6daf2abbd)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.034192 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.053551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:34.098196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d46d5f0032460c8417f8f6daf2abbd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.099127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d46d5f0032460c8417f8f6daf2abbd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.100032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d46d5f0032460c8417f8f6daf2abbd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.110852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d46d5f0032460c8417f8f6daf2abbd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.124757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-4d3c2b41-de67-4976-a5ed-edef09dedeb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d46d5f0032460c8417f8f6daf2abbd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.137167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1257/2508] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:33 2022] DELETE /identity/v3/users/67d46d5f0032460c8417f8f6daf2abbd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:34.327299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.328963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:34.343599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e27cc4-fc0f-4357-a3f4-e5118f6543a3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.402770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.403558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.403836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.404145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] PATH_INFO: `/v3/projects/a84c540dd163496f91bc331639ad61fa` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.405598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84c540dd163496f91bc331639ad61fa)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.407535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.454793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84c540dd163496f91bc331639ad61fa', 'request_id': 'req-81976c0e-74ae-4055-8f67-4babca59ebf2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.460923 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84c540dd163496f91bc331639ad61fa', 'request_id': 'req-81976c0e-74ae-4055-8f67-4babca59ebf2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.467290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84c540dd163496f91bc331639ad61fa', 'request_id': 'req-81976c0e-74ae-4055-8f67-4babca59ebf2'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.481089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] The token cache is being invalidate because project a84c540dd163496f91bc331639ad61fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:34.481402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.482032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-81976c0e-74ae-4055-8f67-4babca59ebf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:34.483771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1252/2509] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:34 2022] DELETE /identity/v3/projects/a84c540dd163496f91bc331639ad61fa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:34.488917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.489433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.489560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.507575 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:34.508842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:34.578968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1258/2510] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:07:34.587242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.587554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.587832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.608138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:34.609368 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:34.618802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1253/2511] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:34.623585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.637853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e5dde-9a9e-427d-bafd-fbc74eb6f54c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.646803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.647796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.648232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.648565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.649695 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:07:34.650151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.652722 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.665827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1259/2512] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:07:34.670277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.679594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c788adb-a9ef-48d9-aacb-080ae7ff422b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.684490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.685483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.685816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.686110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.686817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.688919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.718650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1254/2513] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:34 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:34.723455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.740463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a83d5f-1577-4f6d-b833-0e6560f580db admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.804387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.805057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.805339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.805614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.806303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.808275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.831550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1260/2514] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:34 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:34.836384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.847362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7daaf-1c03-4674-aa03-5a3ef3bfe45b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.853183 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.853878 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.854153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.854420 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.855145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.856778 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.869041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1255/2515] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:34.874581 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.886816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009f77-48d3-45c4-a907-d2a22222127a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.892548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3884cc-0ce4-497f-8252-60ae0be2cbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.893208 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3884cc-0ce4-497f-8252-60ae0be2cbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.893485 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3884cc-0ce4-497f-8252-60ae0be2cbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.893781 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3884cc-0ce4-497f-8252-60ae0be2cbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.894494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3884cc-0ce4-497f-8252-60ae0be2cbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.896127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3884cc-0ce4-497f-8252-60ae0be2cbd0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.901576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:34.911293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1261/2516] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:07:34.913402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:34.924361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e53b5-d5fb-430a-8f04-ff620461d683 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:34.929205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:34.930078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.930436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.930713 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] PATH_INFO: `/v3/projects/984bc8b570b1445d901ac3ec45bea990/users/1bf5c2d81c1c4ddf943dbac18255275d/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:34.945506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=984bc8b570b1445d901ac3ec45bea990, user_id=1bf5c2d81c1c4ddf943dbac18255275d, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:34.948181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:34.961803 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1256/2517] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:34 2022] PUT /identity/v3/projects/984bc8b570b1445d901ac3ec45bea990/users/1bf5c2d81c1c4ddf943dbac18255275d/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:34.979191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:34.979488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:34.979765 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.008593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] MFA Rules not processed for user `1bf5c2d81c1c4ddf943dbac18255275d`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:35.009800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:35.078154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1262/2518] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:35.082865 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.092557 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:35.146859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b71f99-919f-428a-a10c-61ea83ca5c4e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.152589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.153239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.153519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.153790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.154594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.156745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.186618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1257/2519] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:35 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:35.190692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.201536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e759be8-0d81-4624-ba9c-f121ff7bde32 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.260983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.261675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.261950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.262220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.262905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.264984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.290830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1263/2520] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:35 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:35.292078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.303272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f08b0-31fc-4bf5-9703-7bd4539ad819 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.308519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.309197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.309528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.309945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.311184 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.313245 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.325194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1258/2521] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:35.329370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.339360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bf6ab-4e84-4873-8467-a935093dd092 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.344036 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.344931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.345206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.345474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] PATH_INFO: `/v3/projects/4979211b43ee4573afbd87349d77107a/users/988735d916144e54841095627af5e57d/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.361375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4979211b43ee4573afbd87349d77107a, user_id=988735d916144e54841095627af5e57d, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.364067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.380600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1264/2522] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:35 2022] PUT /identity/v3/projects/4979211b43ee4573afbd87349d77107a/users/988735d916144e54841095627af5e57d/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:35.384631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.394431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386884b1-b43b-44b2-b91b-739d48dc975b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.452304 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871c08d-da68-4b9b-9184-8a8158e97f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.452958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871c08d-da68-4b9b-9184-8a8158e97f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.453238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871c08d-da68-4b9b-9184-8a8158e97f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.453517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871c08d-da68-4b9b-9184-8a8158e97f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.454248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871c08d-da68-4b9b-9184-8a8158e97f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.455856 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871c08d-da68-4b9b-9184-8a8158e97f57 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.466462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1259/2523] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:35.471275 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.481918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8285af0-f29c-4513-9631-177ff58bec1c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.487748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.488672 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.488953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.489490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] PATH_INFO: `/v3/projects/4979211b43ee4573afbd87349d77107a/users/988735d916144e54841095627af5e57d/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.491386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4979211b43ee4573afbd87349d77107a, user_id=988735d916144e54841095627af5e57d, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.494069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.505792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1265/2524] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:35 2022] PUT /identity/v3/projects/4979211b43ee4573afbd87349d77107a/users/988735d916144e54841095627af5e57d/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:35.516800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.517091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.517369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.551670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] MFA Rules not processed for user `988735d916144e54841095627af5e57d`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:35.552894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:35.621295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1260/2525] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:35.632154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:35.633981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:35.679292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:35.692431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a91a8-99d0-46bf-9aa2-2e79c52f31bd admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:35.696437 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376b19b-1b73-4a4b-898a-36ce8fe76b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:35.696959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376b19b-1b73-4a4b-898a-36ce8fe76b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:35.697236 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376b19b-1b73-4a4b-898a-36ce8fe76b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:35.697509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376b19b-1b73-4a4b-898a-36ce8fe76b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:35.708101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376b19b-1b73-4a4b-898a-36ce8fe76b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:35.709576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376b19b-1b73-4a4b-898a-36ce8fe76b30 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:35.720414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1266/2526] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:37.129517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:37.133788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:37.198439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945f3ff-4ce3-4fd4-9318-081791d1222f None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:37.203279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:37.204198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:37.204514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:37.204782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] PATH_INFO: `/v3/users/1bf5c2d81c1c4ddf943dbac18255275d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:37.206086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf5c2d81c1c4ddf943dbac18255275d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:37.207983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:37.266068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf5c2d81c1c4ddf943dbac18255275d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.266729 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf5c2d81c1c4ddf943dbac18255275d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.278507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf5c2d81c1c4ddf943dbac18255275d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.279955 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf5c2d81c1c4ddf943dbac18255275d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.306693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf5c2d81c1c4ddf943dbac18255275d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.319548 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1261/2527] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:37 2022] DELETE /identity/v3/users/1bf5c2d81c1c4ddf943dbac18255275d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:37.323850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:37.324322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:37.324557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:37.343499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:37.344819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:37.407393 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1267/2528] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:37.417384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:37.418952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:37.489191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7e72-2459-4080-857e-762820d26038 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:37.496485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:37.497508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:37.498004 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:37.498508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:37.508880 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:37.510338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:37.520464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1262/2529] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:37.806863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:37.808345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:37.822396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc2376-d037-4296-978f-012de3ca7d52 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:37.829055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:37.831502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:37.832291 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:37.832531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] PATH_INFO: `/v3/projects/984bc8b570b1445d901ac3ec45bea990` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:37.834468 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=984bc8b570b1445d901ac3ec45bea990)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:37.837294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:37.895265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984bc8b570b1445d901ac3ec45bea990', 'request_id': 'req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.902353 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984bc8b570b1445d901ac3ec45bea990', 'request_id': 'req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.912116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984bc8b570b1445d901ac3ec45bea990', 'request_id': 'req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.925264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] The token cache is being invalidate because project 984bc8b570b1445d901ac3ec45bea990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:37.925549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.926525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:37.928905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1268/2530] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:37 2022] DELETE /identity/v3/projects/984bc8b570b1445d901ac3ec45bea990 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:37.934875 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:37.948563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f65a8-2a3c-409e-83c4-b3372105cbd1 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:37.973970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:38.019084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:38.019893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.020181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.020456 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] PATH_INFO: `/v3/users/988735d916144e54841095627af5e57d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.021751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=988735d916144e54841095627af5e57d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:38.023682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:38.082975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988735d916144e54841095627af5e57d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.083758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988735d916144e54841095627af5e57d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.095754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988735d916144e54841095627af5e57d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.096514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988735d916144e54841095627af5e57d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.110472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-46a472f9-44c3-4445-99d6-bd1550cdfd90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988735d916144e54841095627af5e57d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.124396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1263/2531] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:37 2022] DELETE /identity/v3/users/988735d916144e54841095627af5e57d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:38.441876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:38.443332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:38.457255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12bd8a-0178-4b06-a1cf-029873bb2b0a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:38.529576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:38.530580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.530943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.532401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] PATH_INFO: `/v3/projects/4979211b43ee4573afbd87349d77107a` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.532663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4979211b43ee4573afbd87349d77107a)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:38.534981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:38.550316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:38.599429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4979211b43ee4573afbd87349d77107a', 'request_id': 'req-272c8fb4-bfe7-4f66-840f-b98c23355590'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.605795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4979211b43ee4573afbd87349d77107a', 'request_id': 'req-272c8fb4-bfe7-4f66-840f-b98c23355590'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.615400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4979211b43ee4573afbd87349d77107a', 'request_id': 'req-272c8fb4-bfe7-4f66-840f-b98c23355590'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.639130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] The token cache is being invalidate because project 4979211b43ee4573afbd87349d77107a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:38.639423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.640128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-272c8fb4-bfe7-4f66-840f-b98c23355590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:38.642025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1269/2532] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:38 2022] DELETE /identity/v3/projects/4979211b43ee4573afbd87349d77107a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:38.647477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.648080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.648208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.673774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:38.674983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:38.738208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1264/2533] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:38.747500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.748180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.748456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.771304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:38.772572 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:38.785215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1270/2534] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:07:38.786611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:38.801734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ae07b-62d2-4fd3-9d7b-434932602383 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:38.806161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:38.806817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.807094 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.807366 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.808128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:07:38.808430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:38.810329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:38.822426 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1265/2535] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:07:38.827064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:38.836586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc2f8-d101-4f21-9e1b-10af3f7cb45a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:38.841413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:38.842248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.842533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.842870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.843523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:38.845451 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:38.872731 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1271/2536] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:38 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:38.877166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:38.887900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa5fda-7d96-43f6-8e59-6381e2f549a9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:38.918945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:38.954066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:38.954993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:38.955375 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:38.955716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:38.956743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:38.959570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:38.986920 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1266/2537] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:38 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:38.991323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.004600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477bfa0-c7f8-4a1c-a435-ed5e2b29e18f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.010376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.011052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.011331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.011626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.012399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.014112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.025301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1272/2538] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:39.029626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.040422 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c7737-7187-4292-a904-a7b1012b4615 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.046571 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c6a03-db38-481f-8272-4a2fdb9e12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.047238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c6a03-db38-481f-8272-4a2fdb9e12a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.047515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c6a03-db38-481f-8272-4a2fdb9e12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.047783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c6a03-db38-481f-8272-4a2fdb9e12a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.048871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c6a03-db38-481f-8272-4a2fdb9e12a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.050751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c6a03-db38-481f-8272-4a2fdb9e12a7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.063796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1267/2539] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:39.068308 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.080954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe12fe6-a6d1-4803-9843-8db9228714ed admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.086844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.087738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.088055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.088369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] PATH_INFO: `/v3/projects/6518bfc998f743de97fa55bcbb6dbef1/users/0a88cbb1fcab4096ab290f4984043f99/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.104082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6518bfc998f743de97fa55bcbb6dbef1, user_id=0a88cbb1fcab4096ab290f4984043f99, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.106768 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.120863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1273/2540] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:39 2022] PUT /identity/v3/projects/6518bfc998f743de97fa55bcbb6dbef1/users/0a88cbb1fcab4096ab290f4984043f99/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:39.131723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.132055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.132360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.164041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] MFA Rules not processed for user `0a88cbb1fcab4096ab290f4984043f99`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:39.165376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:39.241607 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1268/2541] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:39.247167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.314034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07221cd-2e2e-4b4b-ab68-0b172672f974 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.319181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.319841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.320150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.320422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.321114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.323044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.350690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1274/2542] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:39 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:39.355339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.367253 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a0a83-eb0a-44ab-ae40-e17b5a1bf829 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.424421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.425031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.425311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.425591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.426291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.428249 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.452145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1269/2543] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:39 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:39.457723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.468111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624db79-4bf5-4bbb-95d2-0c6c05b45be2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.472780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.473436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.473712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.473981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.474703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.476343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.487070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1275/2544] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:39.491524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.502117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffeeae-9bf6-47d1-8ca4-2f24f157a684 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.507227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.508338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.508595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.508876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] PATH_INFO: `/v3/projects/60a6fca310cb4c8fbdb9ee6dc9e1765c/users/063096081cd24780a1011088912bec5e/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.523844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a6fca310cb4c8fbdb9ee6dc9e1765c, user_id=063096081cd24780a1011088912bec5e, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.526522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.541229 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1270/2545] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:39 2022] PUT /identity/v3/projects/60a6fca310cb4c8fbdb9ee6dc9e1765c/users/063096081cd24780a1011088912bec5e/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:39.545814 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.555702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f0ce5-5ae3-4dda-8925-e9a65002d49a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.560800 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:39.612857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f161891-ac6f-4193-aad5-ea43a949f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.613527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f161891-ac6f-4193-aad5-ea43a949f4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.613806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f161891-ac6f-4193-aad5-ea43a949f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.614080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f161891-ac6f-4193-aad5-ea43a949f4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.614805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f161891-ac6f-4193-aad5-ea43a949f4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.616438 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f161891-ac6f-4193-aad5-ea43a949f4e5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.626188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1276/2546] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:39.630461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.637412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:39.641551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83269-0767-47a7-bf53-6dfe785ecc7e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.647429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.648367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.648934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.648934 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] PATH_INFO: `/v3/projects/60a6fca310cb4c8fbdb9ee6dc9e1765c/users/063096081cd24780a1011088912bec5e/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.650897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a6fca310cb4c8fbdb9ee6dc9e1765c, user_id=063096081cd24780a1011088912bec5e, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.653611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.664174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1271/2547] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:39 2022] PUT /identity/v3/projects/60a6fca310cb4c8fbdb9ee6dc9e1765c/users/063096081cd24780a1011088912bec5e/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:39.674603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.674897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.675173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.702784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] MFA Rules not processed for user `063096081cd24780a1011088912bec5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:39.704277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:39.774639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1277/2548] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:39.786179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.788105 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:39.849409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbfc00-2666-415d-af14-d7d6dd1bc7ec admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.853220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde99cb2-9f3b-4ae6-83a1-35ef76f89654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.853871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99cb2-9f3b-4ae6-83a1-35ef76f89654 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.854152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99cb2-9f3b-4ae6-83a1-35ef76f89654 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.854427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99cb2-9f3b-4ae6-83a1-35ef76f89654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.863315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde99cb2-9f3b-4ae6-83a1-35ef76f89654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.864741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde99cb2-9f3b-4ae6-83a1-35ef76f89654 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:39.873928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1272/2549] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:39.902509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:39.976519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813031-2a83-4a85-bb5f-0bc76a76f4c9 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:39.981481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:39.982343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:39.982632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:39.982901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] PATH_INFO: `/v3/users/0a88cbb1fcab4096ab290f4984043f99` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:39.984162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a88cbb1fcab4096ab290f4984043f99)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:39.986085 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:40.027450 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:40.045261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a88cbb1fcab4096ab290f4984043f99'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.045942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a88cbb1fcab4096ab290f4984043f99'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.046563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a88cbb1fcab4096ab290f4984043f99'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.054204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a88cbb1fcab4096ab290f4984043f99'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.067400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a88cbb1fcab4096ab290f4984043f99'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.080663 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1278/2550] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:39 2022] DELETE /identity/v3/users/0a88cbb1fcab4096ab290f4984043f99 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:40.085394 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:40.085678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:40.085953 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:40.104807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:40.105999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:40.167899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1273/2551] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:40.179078 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:40.180589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:40.239324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595db23b-e5c1-4094-ac39-26e6133c858c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:40.243799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:40.244477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:40.244754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:40.245028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:40.255260 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:40.256734 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:40.267347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1279/2552] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:40.550746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:40.552223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:40.566497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c1d26-33f5-4d1d-bebf-e5ce36bf6874 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:40.572113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:40.573053 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:40.573327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:40.573603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] PATH_INFO: `/v3/projects/6518bfc998f743de97fa55bcbb6dbef1` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:40.574901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6518bfc998f743de97fa55bcbb6dbef1)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:40.577024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:40.638233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6518bfc998f743de97fa55bcbb6dbef1', 'request_id': 'req-467045b2-4931-4849-a4ab-940a7dffee30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.643656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6518bfc998f743de97fa55bcbb6dbef1', 'request_id': 'req-467045b2-4931-4849-a4ab-940a7dffee30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.649985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6518bfc998f743de97fa55bcbb6dbef1', 'request_id': 'req-467045b2-4931-4849-a4ab-940a7dffee30'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.663356 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] The token cache is being invalidate because project 6518bfc998f743de97fa55bcbb6dbef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:40.663637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.664279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.665935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1274/2553] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:40 2022] DELETE /identity/v3/projects/6518bfc998f743de97fa55bcbb6dbef1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:40.670194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:40.680263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5e1bb-8e15-4576-a83a-060936805749 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:40.736832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:40.737694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:40.737979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:40.738247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] PATH_INFO: `/v3/users/063096081cd24780a1011088912bec5e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:40.739460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=063096081cd24780a1011088912bec5e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:40.741399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:40.793886 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063096081cd24780a1011088912bec5e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.794597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063096081cd24780a1011088912bec5e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.795254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063096081cd24780a1011088912bec5e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.803400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063096081cd24780a1011088912bec5e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.804679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:40.817667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-3c5c8d9f-455e-4276-8081-8e336d8b91bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063096081cd24780a1011088912bec5e'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:40.829922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1280/2554] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:40 2022] DELETE /identity/v3/users/063096081cd24780a1011088912bec5e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:41.111095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.112658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:41.126230 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467045b2-4931-4849-a4ab-940a7dffee30 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.140278 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:41.187786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.188601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.188884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.189155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] PATH_INFO: `/v3/projects/60a6fca310cb4c8fbdb9ee6dc9e1765c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.190332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a6fca310cb4c8fbdb9ee6dc9e1765c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.192287 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.240584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a6fca310cb4c8fbdb9ee6dc9e1765c', 'request_id': 'req-77433af4-103d-4e33-a2d2-ae5b87cd00e4'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:41.246019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a6fca310cb4c8fbdb9ee6dc9e1765c', 'request_id': 'req-77433af4-103d-4e33-a2d2-ae5b87cd00e4'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:41.252214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a6fca310cb4c8fbdb9ee6dc9e1765c', 'request_id': 'req-77433af4-103d-4e33-a2d2-ae5b87cd00e4'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:41.265752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] The token cache is being invalidate because project 60a6fca310cb4c8fbdb9ee6dc9e1765c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:41.266042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:41.266653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-77433af4-103d-4e33-a2d2-ae5b87cd00e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:41.268262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1275/2555] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:41 2022] DELETE /identity/v3/projects/60a6fca310cb4c8fbdb9ee6dc9e1765c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:41.273296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.273590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.273864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.292496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:41.293810 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:41.355283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1281/2556] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:41.364197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.364481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.364755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.383823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:41.385054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:41.394514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1276/2557] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:41.398970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.409351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5341fd-429b-45f9-9d74-7d185daf67d0 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.415493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.416179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.416454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.416708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.417452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:07:41.417758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.419364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.431842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1282/2558] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:07:41.436836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.446603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dd6ad-ea1b-4a33-86bb-327100b5ca33 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.452502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.453150 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.453424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.453757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.454446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.456386 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.484678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1277/2559] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:41.489212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.499550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d3f3-333f-4cf4-ae80-8a8b44327d38 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.560532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.561442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.561727 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.562005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.562716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.564715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.587076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1283/2560] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:41.591210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.603500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15c7a-a4bd-4da5-bad4-549da34c0317 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.609835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.610540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.610809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.611084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.611947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.613625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.625244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1278/2561] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:41.629442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.639794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eecddb-71e5-4476-8c49-efcfeeb10daa admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.644832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3a7d3-d86b-45a1-8395-36f956e45b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.645508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a7d3-d86b-45a1-8395-36f956e45b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.645790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a7d3-d86b-45a1-8395-36f956e45b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.646069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a7d3-d86b-45a1-8395-36f956e45b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.646794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3a7d3-d86b-45a1-8395-36f956e45b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.648408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3a7d3-d86b-45a1-8395-36f956e45b42 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.658955 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1284/2562] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:41.663736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.676156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ec9cd-8217-48e8-9ed2-7d9009f50fe4 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.682112 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.683016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.683294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.683566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] PATH_INFO: `/v3/projects/e26981af6c134e16aadf32e6d62bbca9/users/f0f3eaa6c4c147638ff0e8cac306769c/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.699504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26981af6c134e16aadf32e6d62bbca9, user_id=f0f3eaa6c4c147638ff0e8cac306769c, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.702220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.716503 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1279/2563] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:41 2022] PUT /identity/v3/projects/e26981af6c134e16aadf32e6d62bbca9/users/f0f3eaa6c4c147638ff0e8cac306769c/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:41.727298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.727806 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.727937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.754235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] MFA Rules not processed for user `f0f3eaa6c4c147638ff0e8cac306769c`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:41.755442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:41.782791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:41.823808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1285/2564] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:41.828752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.879476 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:41.900339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5e32c-7d5b-4acc-be82-1c39a2f1f739 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:41.906933 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:41.907630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:41.907930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:41.908208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:41.908900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:41.910881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:41.943839 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1280/2565] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:41.945061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:41.960820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5967a-83c9-4b5d-92fc-6d7a36013c1c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.016963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.017677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.017951 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.018224 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.018922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.020891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.041253 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1286/2566] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:41 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:42.045344 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:42.055646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46459323-2863-4414-b43a-c23bd41b8ddc admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.060901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.061583 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.061863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.062135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.062859 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.064490 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.075148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1281/2567] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:42.079554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:42.090112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583d396-f6c1-4be2-9ae5-65089f0fdc49 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.095798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.096720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.096998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.097268 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] PATH_INFO: `/v3/projects/069caf3c9eeb476fa70020c18c51d19e/users/9cc13975be604ed0b897a4143709a680/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.112815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=069caf3c9eeb476fa70020c18c51d19e, user_id=9cc13975be604ed0b897a4143709a680, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.115494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.129856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1287/2568] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:42 2022] PUT /identity/v3/projects/069caf3c9eeb476fa70020c18c51d19e/users/9cc13975be604ed0b897a4143709a680/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:42.133954 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:42.144285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb5077-001d-4b78-9285-b7d8aaa66b63 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.201742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5581ca-59e7-4023-8c47-8c822e0a77fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.202281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5581ca-59e7-4023-8c47-8c822e0a77fa admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.202774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5581ca-59e7-4023-8c47-8c822e0a77fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.202993 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5581ca-59e7-4023-8c47-8c822e0a77fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.203772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5581ca-59e7-4023-8c47-8c822e0a77fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.205405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5581ca-59e7-4023-8c47-8c822e0a77fa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.218414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1282/2569] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:42.222024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:42.232972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05ae1-b2af-4b52-a062-9f0ca350e7cb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.237756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.238641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.238924 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.239199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] PATH_INFO: `/v3/projects/069caf3c9eeb476fa70020c18c51d19e/users/9cc13975be604ed0b897a4143709a680/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.241201 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=069caf3c9eeb476fa70020c18c51d19e, user_id=9cc13975be604ed0b897a4143709a680, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.243857 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.255027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1288/2570] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:42 2022] PUT /identity/v3/projects/069caf3c9eeb476fa70020c18c51d19e/users/9cc13975be604ed0b897a4143709a680/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:42.267349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.267655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.267988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.300083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] MFA Rules not processed for user `9cc13975be604ed0b897a4143709a680`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:42.301301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:42.371981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1283/2571] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:42.383175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:42.385248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:42.447490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be7c29-3841-4fd4-ba4d-6fe88995f92f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.452517 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.453100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.453382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.453649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.463363 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.464839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.474176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1289/2572] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:42.520202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:42.521688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:42.534142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500856d6-be17-4939-96d2-ad4824a4d0d3 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:42.538477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6c8a9-6a1d-42cf-b92c-c7329627e29c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:42.539148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6c8a9-6a1d-42cf-b92c-c7329627e29c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:42.539429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6c8a9-6a1d-42cf-b92c-c7329627e29c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:42.539736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6c8a9-6a1d-42cf-b92c-c7329627e29c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:42.581199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:42.604807 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6c8a9-6a1d-42cf-b92c-c7329627e29c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:42.606272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6c8a9-6a1d-42cf-b92c-c7329627e29c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:42.615351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1284/2573] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:45.947652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:45.975230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:46.014103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a02b9-63eb-4066-b1c7-05d6eda22491 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:46.019146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:46.020061 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:46.020337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:46.020601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] PATH_INFO: `/v3/users/f0f3eaa6c4c147638ff0e8cac306769c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:46.021797 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f3eaa6c4c147638ff0e8cac306769c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:46.023729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:46.075917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f3eaa6c4c147638ff0e8cac306769c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.076849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f3eaa6c4c147638ff0e8cac306769c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.077443 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f3eaa6c4c147638ff0e8cac306769c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.085730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f3eaa6c4c147638ff0e8cac306769c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.097647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f3eaa6c4c147638ff0e8cac306769c'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.109634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1290/2574] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:45 2022] DELETE /identity/v3/users/f0f3eaa6c4c147638ff0e8cac306769c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:46.114263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:46.114728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:46.114824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:46.132768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:46.133987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:46.197892 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1285/2575] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:46.208299 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:46.209811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:46.271115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbb88-766c-47b5-9277-7fe5bd324729 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:46.275586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:46.276278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:46.276563 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:46.276837 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:46.288957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:46.290412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:46.300979 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1291/2576] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:46.586425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:46.588295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:46.602990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a861a9f-54a0-4094-9543-76c3f54b23db None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:46.608913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:46.609873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:46.610137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:46.610532 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] PATH_INFO: `/v3/projects/e26981af6c134e16aadf32e6d62bbca9` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:46.611793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26981af6c134e16aadf32e6d62bbca9)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:46.613830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:46.664884 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26981af6c134e16aadf32e6d62bbca9', 'request_id': 'req-eb7769fd-1746-4cac-af19-85faec80c191'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.670542 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26981af6c134e16aadf32e6d62bbca9', 'request_id': 'req-eb7769fd-1746-4cac-af19-85faec80c191'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.681894 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26981af6c134e16aadf32e6d62bbca9', 'request_id': 'req-eb7769fd-1746-4cac-af19-85faec80c191'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.695685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] The token cache is being invalidate because project e26981af6c134e16aadf32e6d62bbca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:46.696002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.696629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.698332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1286/2577] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:46 2022] DELETE /identity/v3/projects/e26981af6c134e16aadf32e6d62bbca9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:46.702626 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:46.712745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1b8c8-c105-4b1a-b9c3-1a62604f368c service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:46.754396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:46.773423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:46.774322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:46.774601 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:46.774869 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] PATH_INFO: `/v3/users/9cc13975be604ed0b897a4143709a680` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:46.776127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc13975be604ed0b897a4143709a680)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:46.778039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:46.829278 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc13975be604ed0b897a4143709a680'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.829962 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc13975be604ed0b897a4143709a680'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.830585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc13975be604ed0b897a4143709a680'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.839848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc13975be604ed0b897a4143709a680'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.852349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-727e10b1-9f3e-46a6-8bd9-721bcea8e394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc13975be604ed0b897a4143709a680'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:46.865096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1292/2578] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:46 2022] DELETE /identity/v3/users/9cc13975be604ed0b897a4143709a680 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:47.158299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.160005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:47.178649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7769fd-1746-4cac-af19-85faec80c191 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.237238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.238042 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.238321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.238600 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] PATH_INFO: `/v3/projects/069caf3c9eeb476fa70020c18c51d19e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.239784 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069caf3c9eeb476fa70020c18c51d19e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.241734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.244050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:47.291604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069caf3c9eeb476fa70020c18c51d19e', 'request_id': 'req-bdf399df-ddba-4110-869b-68953be27612'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:47.297657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069caf3c9eeb476fa70020c18c51d19e', 'request_id': 'req-bdf399df-ddba-4110-869b-68953be27612'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:47.304678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069caf3c9eeb476fa70020c18c51d19e', 'request_id': 'req-bdf399df-ddba-4110-869b-68953be27612'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:47.319929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] The token cache is being invalidate because project 069caf3c9eeb476fa70020c18c51d19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:47.320214 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:47.320912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bdf399df-ddba-4110-869b-68953be27612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:47.322454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1287/2579] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:47 2022] DELETE /identity/v3/projects/069caf3c9eeb476fa70020c18c51d19e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:47.327820 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.328341 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.328467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.347172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:47.348427 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:47.410003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1293/2580] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:47.418102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.418408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.418685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.438294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:47.439488 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:47.447734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1288/2581] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:47.453621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.464343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d003d5-2ec5-453c-8bdc-1a25a98e51a4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.470589 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.471251 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.471528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.471792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.472553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:07:47.472852 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.474502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.486925 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1294/2582] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:07:47.491461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.501225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc5260-84cc-4a30-b309-4e2416c95249 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.506637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.507288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.507563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.507832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.508549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.510459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.541328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1289/2583] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:47 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:47.546823 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.557096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645e2d-aa81-435c-932b-2abb3c8cd137 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.603623 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:47.617314 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.618004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.618483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.618985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.619691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.621708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.645642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1295/2584] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:47 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:47.650075 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.661078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145b588-a357-4f4f-8912-2c6533e6cc22 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.666436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.667073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.667354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.667630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.668414 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.670169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.681006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1290/2585] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:07:47.685658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.696423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cb3f2-2bc2-471f-a829-3baff653bc51 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.702588 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39d7e6-21fa-46fa-8aa7-000af7c4736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.703369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39d7e6-21fa-46fa-8aa7-000af7c4736c admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.703785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39d7e6-21fa-46fa-8aa7-000af7c4736c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.704107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39d7e6-21fa-46fa-8aa7-000af7c4736c admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.704855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39d7e6-21fa-46fa-8aa7-000af7c4736c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.706488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39d7e6-21fa-46fa-8aa7-000af7c4736c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.717466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1296/2586] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:47.721937 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.732539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3b56-957f-4f37-b7f7-0aeac3cbf7ee admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.739388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.740342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.740626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.740901 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] PATH_INFO: `/v3/projects/30ebacba44c7402196b9cbd881529a52/users/c53dbfae09b24cdb8f23f376700fd48f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.757588 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ebacba44c7402196b9cbd881529a52, user_id=c53dbfae09b24cdb8f23f376700fd48f, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.760251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:47.777284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1291/2587] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:47 2022] PUT /identity/v3/projects/30ebacba44c7402196b9cbd881529a52/users/c53dbfae09b24cdb8f23f376700fd48f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:47.788197 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.788480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.788755 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.816181 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] MFA Rules not processed for user `c53dbfae09b24cdb8f23f376700fd48f`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:47.817628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:47.887828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1297/2588] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:47.895051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:47.962565 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc69d1c-cb4d-41dd-a686-c5e99e81c54c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:47.968338 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:47.968984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:47.969258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:47.969528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:47.970256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:47.972187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.001318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1292/2589] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:47 2022] POST /identity/v3/projects => generated 401 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:07:48.005423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.017466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f664aba-2397-4d6c-aaa6-dd4b62f71ab4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.079510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.080550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.080927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.081337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.082368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.084669 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.114736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1298/2590] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:48 2022] POST /identity/v3/users => generated 307 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:48.118913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.130077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f64d9e-a95d-42ad-ad97-a7d5676ed82f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.134995 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.135695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.136005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.136288 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.137025 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.139204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.150688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1293/2591] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:48.155376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.165495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21aba6c-0650-4fe6-b873-4fb02b1b7a5f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.170995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.171926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.172210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.172482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] PATH_INFO: `/v3/projects/97cbef5494064a209ecc837e9c3e7566/users/c1b35155a6244a30b4ea6f1280d814d5/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.188220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cbef5494064a209ecc837e9c3e7566, user_id=c1b35155a6244a30b4ea6f1280d814d5, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.190965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.206472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1299/2592] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:48 2022] PUT /identity/v3/projects/97cbef5494064a209ecc837e9c3e7566/users/c1b35155a6244a30b4ea6f1280d814d5/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:48.210658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.215904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:48.222929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb925d6-06e5-4a44-8a37-05eb41ee597a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.283906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa3ba-26d4-4403-a0b4-18f52b4f3dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.284514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa3ba-26d4-4403-a0b4-18f52b4f3dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.284794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa3ba-26d4-4403-a0b4-18f52b4f3dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.285064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa3ba-26d4-4403-a0b4-18f52b4f3dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.285850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa3ba-26d4-4403-a0b4-18f52b4f3dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.291558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa3ba-26d4-4403-a0b4-18f52b4f3dbe admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.297788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1294/2593] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:48.302211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.313129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c07b52-3e5b-4a1c-a8e6-22cb77336189 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.317913 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.318826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.319106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.319374 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] PATH_INFO: `/v3/projects/97cbef5494064a209ecc837e9c3e7566/users/c1b35155a6244a30b4ea6f1280d814d5/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.321390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cbef5494064a209ecc837e9c3e7566, user_id=c1b35155a6244a30b4ea6f1280d814d5, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.324048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.334320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1300/2594] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:48 2022] PUT /identity/v3/projects/97cbef5494064a209ecc837e9c3e7566/users/c1b35155a6244a30b4ea6f1280d814d5/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:48.404724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.405015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.405591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.441362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] MFA Rules not processed for user `c1b35155a6244a30b4ea6f1280d814d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:48.442597 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:48.520897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1295/2595] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:48.532137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.534080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:48.548366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:48.596457 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2121f22-3e95-415a-80af-7392160cc02a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.600984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.601893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.602252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.602598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.614763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.616293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.625928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1301/2596] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:48.703378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:48.713824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd27bd6-1c43-4703-996e-be440adbc912 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:48.717890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7ec0b-d34d-4a6d-ae5a-4dc00bd29f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:48.718531 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7ec0b-d34d-4a6d-ae5a-4dc00bd29f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:48.718806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7ec0b-d34d-4a6d-ae5a-4dc00bd29f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:48.719076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7ec0b-d34d-4a6d-ae5a-4dc00bd29f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:48.787653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7ec0b-d34d-4a6d-ae5a-4dc00bd29f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:48.789069 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7ec0b-d34d-4a6d-ae5a-4dc00bd29f84 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:48.799510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1296/2597] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:52.950520 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:53.024069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5151f8-0862-4c31-8336-015927f5af28 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:53.030066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:53.032090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:53.032377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:53.032661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] PATH_INFO: `/v3/users/c53dbfae09b24cdb8f23f376700fd48f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:53.033938 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53dbfae09b24cdb8f23f376700fd48f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:53.035916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:53.087177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53dbfae09b24cdb8f23f376700fd48f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.087856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53dbfae09b24cdb8f23f376700fd48f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.088491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53dbfae09b24cdb8f23f376700fd48f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.098830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53dbfae09b24cdb8f23f376700fd48f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.112705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53dbfae09b24cdb8f23f376700fd48f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.126280 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1302/2598] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:52 2022] DELETE /identity/v3/users/c53dbfae09b24cdb8f23f376700fd48f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:53.130837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:53.131311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:53.131450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:53.151516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:53.152816 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:53.174707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=100 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:53.220987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1297/2599] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:53.230821 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:53.232412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:53.268659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:53.292844 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f16191-eac6-46c2-9e18-c610b4ad0004 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:53.297094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:53.297763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:53.298065 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:53.298294 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:53.308528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:53.310315 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:53.320549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1303/2600] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:53.628591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:53.630899 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:53.645008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81a79f-5cfd-4e58-a0ef-ec2b6b072665 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:53.650990 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:53.652055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:53.652470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:53.652814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] PATH_INFO: `/v3/projects/30ebacba44c7402196b9cbd881529a52` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:53.654168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ebacba44c7402196b9cbd881529a52)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:53.656529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:53.706695 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ebacba44c7402196b9cbd881529a52', 'request_id': 'req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.713022 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ebacba44c7402196b9cbd881529a52', 'request_id': 'req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.720349 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ebacba44c7402196b9cbd881529a52', 'request_id': 'req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.735519 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] The token cache is being invalidate because project 30ebacba44c7402196b9cbd881529a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:53.735902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.736498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.738306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1298/2601] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:53 2022] DELETE /identity/v3/projects/30ebacba44c7402196b9cbd881529a52 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:53.742324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:53.752576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74407-aa84-490b-bb41-5ba006796f17 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:53.819179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:53.820597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:53.820987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:53.821361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] PATH_INFO: `/v3/users/c1b35155a6244a30b4ea6f1280d814d5` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:53.823179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b35155a6244a30b4ea6f1280d814d5)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:53.826281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:53.881232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b35155a6244a30b4ea6f1280d814d5'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.882005 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b35155a6244a30b4ea6f1280d814d5'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.882624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b35155a6244a30b4ea6f1280d814d5'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.891737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b35155a6244a30b4ea6f1280d814d5'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.904100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-c14b7758-b304-41c9-862e-77ddda7471fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b35155a6244a30b4ea6f1280d814d5'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:53.916624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1304/2602] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:07:53 2022] DELETE /identity/v3/users/c1b35155a6244a30b4ea6f1280d814d5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:54.202323 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.204472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:54.225171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd65ce4-4d52-4756-8a84-a0a4bdb37b8a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.296425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.297305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.297581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.297852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] PATH_INFO: `/v3/projects/97cbef5494064a209ecc837e9c3e7566` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.299071 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97cbef5494064a209ecc837e9c3e7566)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.301031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.359802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:54.361469 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cbef5494064a209ecc837e9c3e7566', 'request_id': 'req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:54.368142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cbef5494064a209ecc837e9c3e7566', 'request_id': 'req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:54.374601 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cbef5494064a209ecc837e9c3e7566', 'request_id': 'req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:54.389141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] The token cache is being invalidate because project 97cbef5494064a209ecc837e9c3e7566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:07:54.389430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:54.390293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-8dd8f7a8-633c-40ed-a036-0a148ea48dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:07:54.391861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1299/2603] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:07:54 2022] DELETE /identity/v3/projects/97cbef5494064a209ecc837e9c3e7566 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:07:54.396927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.397390 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.397510 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.416289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:54.417522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:54.468217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:54.482971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1305/2604] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:54.494096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.494384 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.494664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.515163 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:54.516973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:54.526522 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1300/2605] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:07:54.531066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.541433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214191e-e335-4cf8-93a3-d4ead7176785 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.547293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.547975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.548237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.548504 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.549229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:07:54.549530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.551127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.564003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1306/2606] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:07:54.568578 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.578904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f79692-ed79-43ca-9ef1-61ea49596a96 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.584611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.585268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.585561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.585924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.586628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.588589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.618945 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1301/2607] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:07:54 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:54.624770 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.637794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0b602-d82c-43e0-85df-be55f7022138 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.700907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.701587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.701885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.702120 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.702801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.704846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.728081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1307/2608] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:07:54 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:07:54.732030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.743003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dc443-29b5-4dd2-a659-9e9ed4fbb375 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.748182 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.748860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.749142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.749626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.750873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.752506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.764034 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1302/2609] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:07:54.768786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.779678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8178256-e443-4008-a0cb-1ee7ed5f31c5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.784831 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bac38-0bb5-42ce-a37c-590b4124b949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.785502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bac38-0bb5-42ce-a37c-590b4124b949 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.785779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bac38-0bb5-42ce-a37c-590b4124b949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.786050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bac38-0bb5-42ce-a37c-590b4124b949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.786776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bac38-0bb5-42ce-a37c-590b4124b949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.788715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bac38-0bb5-42ce-a37c-590b4124b949 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.800309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1308/2610] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:07:54.805018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.817145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f9b37-c37a-4c10-8edf-fcc0d7b88955 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:54.825045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:54.826550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.826925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.827673 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] PATH_INFO: `/v3/projects/012dd3b35f2f44c98a5efc3497ed1309/users/4ceacb513b2f4827ab4b438071fdb0b8/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.847703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012dd3b35f2f44c98a5efc3497ed1309, user_id=4ceacb513b2f4827ab4b438071fdb0b8, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:54.850413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:54.864700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1303/2611] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:07:54 2022] PUT /identity/v3/projects/012dd3b35f2f44c98a5efc3497ed1309/users/4ceacb513b2f4827ab4b438071fdb0b8/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:07:54.875910 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:54.876146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:54.876429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:54.905804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] MFA Rules not processed for user `4ceacb513b2f4827ab4b438071fdb0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:07:54.907328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:54.983470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1309/2612] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:07:54.997572 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:54.999293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:55.059005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ddd4-b200-4be2-a372-7de39f71f3e0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:55.063041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:55.063686 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:55.064000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:55.064276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:55.072595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:55.074049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:55.082850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1304/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:07:55.934015 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:07:55.935996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:55.993248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e9c94-ae43-4a8b-bf13-b3cd6e195711 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:07:55.996894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:07:55.998899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:07:55.998899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:07:55.998899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:07:56.000573 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:07:56.009154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:07:56.073972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:07:56.075643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:07:56.091187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1310/2614] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:00.022153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.023842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:00.042708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c806-6a91-40a2-8c6b-a9263a5043b9 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.047472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.048415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.048688 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.048960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] PATH_INFO: `/v3/users/4b1614d524d24ce486741b599268995a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.050186 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1614d524d24ce486741b599268995a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.052306 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.108302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1614d524d24ce486741b599268995a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.109032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1614d524d24ce486741b599268995a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.122938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1614d524d24ce486741b599268995a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.122938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1614d524d24ce486741b599268995a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.130906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:00.132124 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b5b7129b-09a6-4cbb-9d05-d79e24c235ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1614d524d24ce486741b599268995a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.147192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1305/2615] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:00 2022] DELETE /identity/v3/users/4b1614d524d24ce486741b599268995a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:00.334081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.346351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55284153-e1f1-4d43-aa63-9315b098ce9d service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.420030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.420911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.421191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.421537 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] PATH_INFO: `/v3/projects/f43a6e910b6b47c09a859ac798bff5cd` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.422729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43a6e910b6b47c09a859ac798bff5cd)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.424713 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.473185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43a6e910b6b47c09a859ac798bff5cd', 'request_id': 'req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.478906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43a6e910b6b47c09a859ac798bff5cd', 'request_id': 'req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.487752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43a6e910b6b47c09a859ac798bff5cd', 'request_id': 'req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.498631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] The token cache is being invalidate because project f43a6e910b6b47c09a859ac798bff5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:00.498915 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.499548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7d92cd3-6e53-453e-8ff3-9871b2bc18a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:00.501471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1311/2616] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:00 2022] DELETE /identity/v3/projects/f43a6e910b6b47c09a859ac798bff5cd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:00.506293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.506580 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.506858 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.526904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:00.528174 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:00.590766 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1306/2617] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:00.598926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.600263 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.600613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.619887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:00.621172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:00.629706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1312/2618] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:00.634074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.645399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90d780-e50d-4bcf-bc0c-a5293a147d8e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.650545 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.651259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.651669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.652082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.652832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:08:00.653143 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.654768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.673417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1307/2619] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:08:00.677444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.687277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0772-261d-48b6-aa2c-10ead659a543 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.692498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.693155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.693429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.693700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.694400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.696398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.725641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1313/2620] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:00 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:08:00.727118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.739102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56f5be-4ae6-41e8-9e87-18abd6785b82 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.800850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.801510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.803908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.803908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.803908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.804666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.826156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1308/2621] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:00 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:00.830476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.836097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:00.841684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b460-03fe-4351-9afb-2ff85425dc2e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.848864 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.849847 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.850035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.850422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.851424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.853373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.867387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1314/2622] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:08:00.868666 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.878876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c682e2-d80f-47c9-977c-ec50d293625d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.883820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfe9b4-4220-41d3-aa84-7225a861850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.884517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfe9b4-4220-41d3-aa84-7225a861850f admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.884800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfe9b4-4220-41d3-aa84-7225a861850f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.885079 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfe9b4-4220-41d3-aa84-7225a861850f admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.885809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfe9b4-4220-41d3-aa84-7225a861850f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.887408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfe9b4-4220-41d3-aa84-7225a861850f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.897726 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1309/2623] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:00.901957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:00.912708 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee63d19-9b56-417d-9ef5-bb3c20315dab admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:00.918066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:00.918952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.919249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.919492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] PATH_INFO: `/v3/projects/3b971d3c5c5c4cc1ad22101c72d1b2a5/users/8c5acdee445b4460bdbd882556653085/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.935483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b971d3c5c5c4cc1ad22101c72d1b2a5, user_id=8c5acdee445b4460bdbd882556653085, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:00.938723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:00.953686 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1315/2624] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:00 2022] PUT /identity/v3/projects/3b971d3c5c5c4cc1ad22101c72d1b2a5/users/8c5acdee445b4460bdbd882556653085/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:00.964982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:00.965269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:00.965540 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:00.993424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] MFA Rules not processed for user `8c5acdee445b4460bdbd882556653085`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:00.994653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:01.039081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:01.062977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1310/2625] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:01.074309 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:01.076204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:01.134340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee711e8-ee3a-479b-bfd1-96aea813b094 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:01.138776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d996c8b-21d3-46d0-9d58-8c756362b638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:01.139516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d996c8b-21d3-46d0-9d58-8c756362b638 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:01.139894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d996c8b-21d3-46d0-9d58-8c756362b638 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:01.140343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d996c8b-21d3-46d0-9d58-8c756362b638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:01.149917 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d996c8b-21d3-46d0-9d58-8c756362b638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:01.151355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d996c8b-21d3-46d0-9d58-8c756362b638 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:01.160746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1316/2626] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:05.193835 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:05.195471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:05.262152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565dd8d-5e1c-4d0e-a0f9-a577c020fa8e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:05.267301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:05.268188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:05.268463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:05.268715 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] PATH_INFO: `/v3/users/4ceacb513b2f4827ab4b438071fdb0b8` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:05.269989 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ceacb513b2f4827ab4b438071fdb0b8)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:05.271910 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:05.324433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ceacb513b2f4827ab4b438071fdb0b8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.325136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ceacb513b2f4827ab4b438071fdb0b8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.333919 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ceacb513b2f4827ab4b438071fdb0b8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.334561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ceacb513b2f4827ab4b438071fdb0b8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.347096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ceacb513b2f4827ab4b438071fdb0b8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.361021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1311/2627] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:05 2022] DELETE /identity/v3/users/4ceacb513b2f4827ab4b438071fdb0b8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:05.365746 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:05.366205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:05.366383 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:05.390175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:05.391335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:05.454920 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1317/2628] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:05.465716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:05.466282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:05.525746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da614b-59e3-41e3-ba1e-e63e9c142133 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:05.529694 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:05.530335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:05.530610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:05.530879 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:05.541109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:05.542602 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:05.552404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1312/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:05.841463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:05.843166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:05.859772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90395b-11f0-4043-a29b-8280d9ed36a7 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:05.868401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:05.869413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:05.869698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:05.870446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] PATH_INFO: `/v3/projects/012dd3b35f2f44c98a5efc3497ed1309` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:05.871968 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012dd3b35f2f44c98a5efc3497ed1309)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:05.880603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:05.933715 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012dd3b35f2f44c98a5efc3497ed1309', 'request_id': 'req-fc25d3b5-beca-4e2f-aab1-b6d546683879'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.939828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012dd3b35f2f44c98a5efc3497ed1309', 'request_id': 'req-fc25d3b5-beca-4e2f-aab1-b6d546683879'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.948550 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012dd3b35f2f44c98a5efc3497ed1309', 'request_id': 'req-fc25d3b5-beca-4e2f-aab1-b6d546683879'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.952384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:05.965740 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] The token cache is being invalidate because project 012dd3b35f2f44c98a5efc3497ed1309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:05.966127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.966840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-fc25d3b5-beca-4e2f-aab1-b6d546683879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:05.968538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1318/2630] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:05 2022] DELETE /identity/v3/projects/012dd3b35f2f44c98a5efc3497ed1309 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:09.610930 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:09.612691 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:09.622396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:09.626524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383191c9-d70c-4d6a-b63b-5f705189934a service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:09.685982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:09.686883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:09.687156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:09.687395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] PATH_INFO: `/v3/users/8c5acdee445b4460bdbd882556653085` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:09.688615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5acdee445b4460bdbd882556653085)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:09.690642 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:09.741378 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5acdee445b4460bdbd882556653085'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:09.742034 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5acdee445b4460bdbd882556653085'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:09.749526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5acdee445b4460bdbd882556653085'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:09.750158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5acdee445b4460bdbd882556653085'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:09.762169 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5acdee445b4460bdbd882556653085'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:09.775243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1313/2631] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:09 2022] DELETE /identity/v3/users/8c5acdee445b4460bdbd882556653085 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:09.780057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:09.780597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:09.780597 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:09.798474 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:09.799671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:09.860879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1319/2632] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:08:09.871595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:09.873153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:09.932063 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1bcf7-5d08-4011-9ce0-1b598a7d9c5e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:09.936514 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6562e-c990-4b20-b002-8ab66f2902f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:09.937181 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6562e-c990-4b20-b002-8ab66f2902f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:09.937458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6562e-c990-4b20-b002-8ab66f2902f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:09.937746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6562e-c990-4b20-b002-8ab66f2902f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:09.947263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6562e-c990-4b20-b002-8ab66f2902f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:09.948736 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6562e-c990-4b20-b002-8ab66f2902f6 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:09.958876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1314/2633] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:10.243998 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.245495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:10.257842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca824b6-6818-4009-bf92-b4451dd9eea4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.263992 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.264975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.265328 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.265596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] PATH_INFO: `/v3/projects/3b971d3c5c5c4cc1ad22101c72d1b2a5` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.266796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b971d3c5c5c4cc1ad22101c72d1b2a5)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.269323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.325419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b971d3c5c5c4cc1ad22101c72d1b2a5', 'request_id': 'req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:10.332385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b971d3c5c5c4cc1ad22101c72d1b2a5', 'request_id': 'req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:10.340694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b971d3c5c5c4cc1ad22101c72d1b2a5', 'request_id': 'req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:10.351739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] The token cache is being invalidate because project 3b971d3c5c5c4cc1ad22101c72d1b2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:10.352055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:10.352662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-579b3282-7ff1-463c-a4e3-40c3d0cd89f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:10.354193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1320/2634] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:10 2022] DELETE /identity/v3/projects/3b971d3c5c5c4cc1ad22101c72d1b2a5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:10.359121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.359403 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.359668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.379342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:10.380506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:10.401195 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:10.444798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1315/2635] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:10.452870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.453155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.453431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.473165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:10.474367 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:10.483316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1321/2636] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:10.487703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.497887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf78ae-bb52-4a79-8e68-7070655bc905 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.503728 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.504485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.504758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.505024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.505756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:08:10.506057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.507676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.520398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1316/2637] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:08:10.524960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.536684 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5c62-90c5-458d-b878-a77f1b7ee664 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.545700 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.546691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.547088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.547493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.548659 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.552764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.587306 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1322/2638] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:10 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:10.591904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.602960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb917a6-927a-4532-b879-3ab557b84ce7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.666453 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.667073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.667303 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.667575 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.668295 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.670258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.693755 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1317/2639] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:10 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:10.698182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.709107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b51ebf-97e7-453a-b67b-f3b24a26908c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.713981 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.714649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.714933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.715211 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.715983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.717598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.728111 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1323/2640] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:10.732393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.742269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46500dd9-415f-4f23-b32b-371db25d9e97 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.746996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f12f8-c979-4de2-979d-9445e530ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.747649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f12f8-c979-4de2-979d-9445e530ea59 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.747962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f12f8-c979-4de2-979d-9445e530ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.748255 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f12f8-c979-4de2-979d-9445e530ea59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.748963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f12f8-c979-4de2-979d-9445e530ea59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.750561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f12f8-c979-4de2-979d-9445e530ea59 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.760594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1318/2641] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:10.764795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.774792 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129d95-62dc-4912-9728-88cc455771ac admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:10.779959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:10.780848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.781408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.781408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] PATH_INFO: `/v3/projects/b99e31b239d241eabfc90372d9607291/users/f607aaa739fe41f9ba87336efc127bfa/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.797665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99e31b239d241eabfc90372d9607291, user_id=f607aaa739fe41f9ba87336efc127bfa, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:10.800434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:10.821087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1324/2642] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:10 2022] PUT /identity/v3/projects/b99e31b239d241eabfc90372d9607291/users/f607aaa739fe41f9ba87336efc127bfa/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:08:10.835818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:10.836134 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:10.836410 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:10.863561 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] MFA Rules not processed for user `f607aaa739fe41f9ba87336efc127bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:10.864799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:10.934996 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1319/2643] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:10.939692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:10.953785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:11.007200 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40115ce3-aa46-4c2c-925a-e9c3d803632d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.012484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.013145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.013422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.013706 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.014398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.016491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.045003 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1325/2644] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:10 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:11.050461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:11.062151 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819f61-6880-437c-9112-523f7b4d5b25 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.080439 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:11.120380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.121051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.121332 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.121610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.122318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.124281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.144307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1320/2645] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:11 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:11.149732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:11.163532 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f39da4-4801-4586-b2bf-4e6867613c81 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.168434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.169423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.169747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.170081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.171218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.173217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.185140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1326/2646] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:11.189413 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:11.200218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9449dc9-35f5-40ee-a814-334e8da8f946 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.206128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.207020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.207305 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.207574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] PATH_INFO: `/v3/projects/b2c8f059cbbb4ebc8041f8f1ef61f953/users/c8688869c6e94d7bb58bdbd3bc1c83da/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.223518 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c8f059cbbb4ebc8041f8f1ef61f953, user_id=c8688869c6e94d7bb58bdbd3bc1c83da, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.226220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.241404 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1321/2647] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:11 2022] PUT /identity/v3/projects/b2c8f059cbbb4ebc8041f8f1ef61f953/users/c8688869c6e94d7bb58bdbd3bc1c83da/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:11.245489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:11.255613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ad1c8-8b54-42ea-95c6-78050704185c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.315724 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889dca9-c747-47ce-8d14-b62dee61c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.316431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889dca9-c747-47ce-8d14-b62dee61c581 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.316711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889dca9-c747-47ce-8d14-b62dee61c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.316983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889dca9-c747-47ce-8d14-b62dee61c581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.317704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889dca9-c747-47ce-8d14-b62dee61c581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.319300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889dca9-c747-47ce-8d14-b62dee61c581 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.328854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1327/2648] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:11.333164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:11.343541 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8667f-5975-49a2-bf7f-1bee509974bb admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.348598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.349483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.349764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.350033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] PATH_INFO: `/v3/projects/b2c8f059cbbb4ebc8041f8f1ef61f953/users/c8688869c6e94d7bb58bdbd3bc1c83da/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.352109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c8f059cbbb4ebc8041f8f1ef61f953, user_id=c8688869c6e94d7bb58bdbd3bc1c83da, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.354731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.364948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1322/2649] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:11 2022] PUT /identity/v3/projects/b2c8f059cbbb4ebc8041f8f1ef61f953/users/c8688869c6e94d7bb58bdbd3bc1c83da/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:11.377389 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.377599 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.377980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.409277 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] MFA Rules not processed for user `c8688869c6e94d7bb58bdbd3bc1c83da`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:11.410480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:11.481878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1328/2650] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:11.492951 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:11.495027 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:11.564285 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0749-804d-45a7-8b7c-9a5d93f24326 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:11.569008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:11.569655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:11.569979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:11.570247 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:11.631798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:11.633294 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:11.642482 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1323/2651] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:16.318144 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:16.328292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ff033-be78-4fd5-a13c-434a2fff3309 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:16.332677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842cef4-d2aa-46e5-bf69-c140708ef7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:16.333320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842cef4-d2aa-46e5-bf69-c140708ef7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:16.333604 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842cef4-d2aa-46e5-bf69-c140708ef7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:16.333879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842cef4-d2aa-46e5-bf69-c140708ef7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:16.340943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:16.343423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842cef4-d2aa-46e5-bf69-c140708ef7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:16.345194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842cef4-d2aa-46e5-bf69-c140708ef7a1 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:16.355804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1329/2652] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:21.638194 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:26.357022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:36.946216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:36.947986 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:37.035006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d688263-f29b-4195-b75a-b3f69ef86c73 service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:37.040992 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:37.041682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:37.041965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:37.042241 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:37.044200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:37.057471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:37.059395 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:37.070328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1324/2653] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:38.123703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380c6b-8f1a-4ed8-a882-e2e904577992 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:38.124238 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380c6b-8f1a-4ed8-a882-e2e904577992 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:38.124352 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380c6b-8f1a-4ed8-a882-e2e904577992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:38.142416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-05380c6b-8f1a-4ed8-a882-e2e904577992 None None] MFA Rules not processed for user `ead8ee09f8514e75882c877042764a92`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:38.143758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-05380c6b-8f1a-4ed8-a882-e2e904577992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:38.199928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1330/2654] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:38.340471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:38.405361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c7987-55a3-4023-9ab9-22758b13818b service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:38.409917 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:38.410796 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:38.411077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:38.411354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] PATH_INFO: `/v3/users/f607aaa739fe41f9ba87336efc127bfa` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:38.412547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f607aaa739fe41f9ba87336efc127bfa)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:38.414475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:38.464677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f607aaa739fe41f9ba87336efc127bfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:38.465325 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f607aaa739fe41f9ba87336efc127bfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:38.474335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f607aaa739fe41f9ba87336efc127bfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:38.474991 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f607aaa739fe41f9ba87336efc127bfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:38.488122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f607aaa739fe41f9ba87336efc127bfa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:38.501118 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1325/2655] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:38 2022] DELETE /identity/v3/users/f607aaa739fe41f9ba87336efc127bfa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:38.505702 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:38.505996 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:38.506276 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:38.523750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:38.524987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:38.584351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1331/2656] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:38.594505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:38.596397 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:38.607776 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:38.657696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa63e88-c452-42c0-a2f4-cb5555ebd501 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:38.661664 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:38.662322 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:38.662603 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:38.662877 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:38.672820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:38.674359 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:38.685478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1326/2657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:39.097720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:39.107094 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:39.136714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f2c3c-ae09-4773-8a2d-e1d847d30036 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:39.146016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:39.146984 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:39.148179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:39.148736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] PATH_INFO: `/v3/projects/b99e31b239d241eabfc90372d9607291` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:39.152113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99e31b239d241eabfc90372d9607291)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:39.154578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:39.241833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99e31b239d241eabfc90372d9607291', 'request_id': 'req-20781b02-6019-48c7-9db7-93f59f81cc7b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.250422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99e31b239d241eabfc90372d9607291', 'request_id': 'req-20781b02-6019-48c7-9db7-93f59f81cc7b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.262775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99e31b239d241eabfc90372d9607291', 'request_id': 'req-20781b02-6019-48c7-9db7-93f59f81cc7b'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.293039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] The token cache is being invalidate because project b99e31b239d241eabfc90372d9607291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:39.293628 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.295100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.299705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1332/2658] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:39 2022] DELETE /identity/v3/projects/b99e31b239d241eabfc90372d9607291 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:39.305101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:39.329089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633699-699a-4083-9b02-9d2aacbcaf45 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:39.447658 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:39.453864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:39.453864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:39.453864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] PATH_INFO: `/v3/users/c8688869c6e94d7bb58bdbd3bc1c83da` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:39.454655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8688869c6e94d7bb58bdbd3bc1c83da)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:39.461197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:39.557459 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8688869c6e94d7bb58bdbd3bc1c83da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.558606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8688869c6e94d7bb58bdbd3bc1c83da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.573645 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8688869c6e94d7bb58bdbd3bc1c83da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.574599 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8688869c6e94d7bb58bdbd3bc1c83da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.603095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-109bf8cd-4d3a-434e-8213-173f871d68c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8688869c6e94d7bb58bdbd3bc1c83da'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:39.631734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1327/2659] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:39 2022] DELETE /identity/v3/users/c8688869c6e94d7bb58bdbd3bc1c83da => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:08:39.960488 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:39.961937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:39.978426 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20781b02-6019-48c7-9db7-93f59f81cc7b admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.017217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:40.052179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.053069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.053348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.053641 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] PATH_INFO: `/v3/projects/b2c8f059cbbb4ebc8041f8f1ef61f953` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.054882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c8f059cbbb4ebc8041f8f1ef61f953)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.056978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.107377 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c8f059cbbb4ebc8041f8f1ef61f953', 'request_id': 'req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:40.113176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c8f059cbbb4ebc8041f8f1ef61f953', 'request_id': 'req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:40.122281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c8f059cbbb4ebc8041f8f1ef61f953', 'request_id': 'req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:40.137957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] The token cache is being invalidate because project b2c8f059cbbb4ebc8041f8f1ef61f953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:40.138678 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:40.138877 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a1e01c72-1c5a-4fbd-b6e7-efee8abd2af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:40.140660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1333/2660] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:39 2022] DELETE /identity/v3/projects/b2c8f059cbbb4ebc8041f8f1ef61f953 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:40.146926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.147271 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.147757 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.168474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:40.169734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:40.199644 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:40.237798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1328/2661] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:40.247336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.247615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.247929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.268154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:40.269434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:40.279416 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1334/2662] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:40.284101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.294360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e335b-ef12-4fa2-be6a-849ef41ca7f1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.299346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.300048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.300327 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.300633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.301372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:08:40.301677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.303280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.316855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1329/2663] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:08:40.321774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.330907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb625aa9-ca43-4d82-b15f-34efef66d72a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.335529 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.336239 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.336498 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.336766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.337469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.339356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.367583 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1335/2664] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:40.371926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.384436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36f98-1ed3-4d87-ad71-280d23fcf4e7 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.449097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.449939 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.450282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.450758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.451509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.453544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.475133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1330/2665] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:40.481409 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.493249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e155b2f-4aea-4695-b323-d0166f9160b3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.497935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.498615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.498897 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.499172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.500004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.510107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.527034 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1336/2666] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:08:40.530815 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.542465 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c2e84-0e34-4337-8159-f44d4d063dec admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.558604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69910b-9945-435d-8504-5fb546e5be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.559074 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69910b-9945-435d-8504-5fb546e5be7b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.559479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69910b-9945-435d-8504-5fb546e5be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.559817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69910b-9945-435d-8504-5fb546e5be7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.560968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69910b-9945-435d-8504-5fb546e5be7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.562527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69910b-9945-435d-8504-5fb546e5be7b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.574410 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1331/2667] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:40.579963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.591167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd26f-e517-4c82-99d6-99b188407a83 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.596434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.597400 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.597784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.598140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] PATH_INFO: `/v3/projects/671f6acbee494c978a1bab27b06598a6/users/61b1b537531948db89ae358bb406f541/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.615819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671f6acbee494c978a1bab27b06598a6, user_id=61b1b537531948db89ae358bb406f541, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.618493 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.634971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1337/2668] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:40 2022] PUT /identity/v3/projects/671f6acbee494c978a1bab27b06598a6/users/61b1b537531948db89ae358bb406f541/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:40.646741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.646956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.647209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.681994 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] MFA Rules not processed for user `61b1b537531948db89ae358bb406f541`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:40.683516 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:40.759608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1332/2669] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:40.764501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.833834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:40.841478 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf7793-c0e5-42cc-baff-09b4a09cee08 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.849745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.850430 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.851057 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.851621 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.852846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.855296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:40.885878 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1338/2670] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:40.890089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:40.902225 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda7a9-6fa5-4c17-a545-c4aefe7d605e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:40.941690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:40.991670 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:40.992909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:40.993415 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:40.993860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:40.994781 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:40.997205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.022746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1333/2671] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:40 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:08:41.026817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.038313 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d7c08-c0e2-4db8-b1af-1cb2ffb380b6 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.042990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.043673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.043986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.044264 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.044991 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.046590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.057970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1339/2672] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:08:41.062282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.072512 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db7aaa-a193-4f46-b99e-760f2a91b79a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.078098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4191c-5f9d-4dcd-b0dd-d44c3626bdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.079576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4191c-5f9d-4dcd-b0dd-d44c3626bdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.079576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4191c-5f9d-4dcd-b0dd-d44c3626bdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.079823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4191c-5f9d-4dcd-b0dd-d44c3626bdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.080547 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4191c-5f9d-4dcd-b0dd-d44c3626bdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.082319 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4191c-5f9d-4dcd-b0dd-d44c3626bdc3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.093062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1334/2673] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:41.097461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.107414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd214-11ef-4f61-b8ac-0e2e4a4e0a2f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.112894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.113791 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.114073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.114370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] PATH_INFO: `/v3/projects/09bc8d1ac1b84b6a83aea7573bf8d0b3/users/c95826ddf4f84ffb8e91e5a904ec6d1f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.129527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=09bc8d1ac1b84b6a83aea7573bf8d0b3, user_id=c95826ddf4f84ffb8e91e5a904ec6d1f, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.132216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.147671 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1340/2674] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:41 2022] PUT /identity/v3/projects/09bc8d1ac1b84b6a83aea7573bf8d0b3/users/c95826ddf4f84ffb8e91e5a904ec6d1f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:41.159562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.159893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.160165 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.189862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] MFA Rules not processed for user `c95826ddf4f84ffb8e91e5a904ec6d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:41.191038 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:41.279969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1335/2675] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:41.284505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.355828 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a8a8a-5fa3-4139-af25-6d2d0bfe1cba admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.361841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.362533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.362819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.363100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.363819 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.365777 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.396824 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1341/2676] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:41 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:41.400650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.412389 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210ffa-99d8-48e3-9c4a-5c7fa3bace86 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.481535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.481535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.481535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.481535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.482473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.482473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.503956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1336/2677] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:41 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:08:41.508103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.518382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2107-399d-4b32-87e6-f7658b4de369 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.523167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.523836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.524150 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.524425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.525155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.526751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.538068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1342/2678] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:08:41.542311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.552969 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51f6b4-1d47-4cc1-bb7e-b1bf08bc23c0 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.558760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.559667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.559982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.560254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] PATH_INFO: `/v3/projects/85c38ab29f7a4f668648eb9de0f6f724/users/6504083c89a747698a0a2bf9feeececd/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.576048 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c38ab29f7a4f668648eb9de0f6f724, user_id=6504083c89a747698a0a2bf9feeececd, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.578733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.589425 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:41.598088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1337/2679] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:41 2022] PUT /identity/v3/projects/85c38ab29f7a4f668648eb9de0f6f724/users/6504083c89a747698a0a2bf9feeececd/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:41.599644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.611096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b033b-cecc-47be-8473-fa41b1c197e4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.680056 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f674dbf-45c1-4fef-b7d0-706ec197012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.680997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f674dbf-45c1-4fef-b7d0-706ec197012e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.681317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f674dbf-45c1-4fef-b7d0-706ec197012e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.681617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f674dbf-45c1-4fef-b7d0-706ec197012e admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.682411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f674dbf-45c1-4fef-b7d0-706ec197012e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.684596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f674dbf-45c1-4fef-b7d0-706ec197012e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.696940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1343/2680] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:41.700940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.711581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442665b2-c87c-40f2-a8ef-fa0c9a98f338 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.717458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.717970 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.718258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.718551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] PATH_INFO: `/v3/projects/85c38ab29f7a4f668648eb9de0f6f724/users/6504083c89a747698a0a2bf9feeececd/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.720862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c38ab29f7a4f668648eb9de0f6f724, user_id=6504083c89a747698a0a2bf9feeececd, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.723479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.735950 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1338/2681] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:41 2022] PUT /identity/v3/projects/85c38ab29f7a4f668648eb9de0f6f724/users/6504083c89a747698a0a2bf9feeececd/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:41.745511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.745817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.746097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.775132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] MFA Rules not processed for user `6504083c89a747698a0a2bf9feeececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:41.776381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:41.780518 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:41.863805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1344/2682] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:08:41.874313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:41.876818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:41.950527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804b20-39f4-4059-97b7-7c518651434e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:41.955093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:41.956024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:41.956381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:41.956724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:41.967931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:41.969799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:41.983385 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1339/2683] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:42.579853 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:42.581436 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:42.594096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45d297-1f8f-4ffe-afa6-fd8a39daac66 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:42.599489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:42.600178 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:42.600458 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:42.600730 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:42.666453 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:42.667914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:42.676813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1345/2684] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:46.730147 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:46.739529 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfdd65-beee-4e22-8b9f-c6a9f78798ec service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:46.743630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85c94e-775a-47e4-be84-05f1320a063c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:46.744312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85c94e-775a-47e4-be84-05f1320a063c service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:46.744649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85c94e-775a-47e4-be84-05f1320a063c service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:46.744922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85c94e-775a-47e4-be84-05f1320a063c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:46.806393 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85c94e-775a-47e4-be84-05f1320a063c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:46.807962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85c94e-775a-47e4-be84-05f1320a063c service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:46.818279 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1340/2685] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:48.989701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:49.057501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f618f-60aa-467d-8e47-71910b85a449 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:49.062769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:49.063631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:49.063929 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:49.064220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] PATH_INFO: `/v3/users/61b1b537531948db89ae358bb406f541` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:49.065423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b1b537531948db89ae358bb406f541)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:49.067337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:49.113975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:49.122536 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b1b537531948db89ae358bb406f541'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.124020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b1b537531948db89ae358bb406f541'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.124361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b1b537531948db89ae358bb406f541'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.133928 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b1b537531948db89ae358bb406f541'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.147555 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b1b537531948db89ae358bb406f541'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.162524 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1346/2686] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:48 2022] DELETE /identity/v3/users/61b1b537531948db89ae358bb406f541 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:49.167235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:49.167744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:49.167860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:49.189606 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:49.190753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:49.251055 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1341/2687] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:49.260847 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:49.262446 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:49.323081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8802a04-db22-4b20-a7b9-0adf408ee571 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:49.326683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:49.327331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:49.327612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:49.327893 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:49.336762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:49.338196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:49.347580 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1347/2688] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:49.630109 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:49.631693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:49.653263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4418018-b2b4-4c99-9292-378ce5b8acd0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:49.659328 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:49.660577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:49.661594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:49.661913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] PATH_INFO: `/v3/projects/671f6acbee494c978a1bab27b06598a6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:49.663655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=671f6acbee494c978a1bab27b06598a6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:49.666049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:49.694944 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:49.720493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671f6acbee494c978a1bab27b06598a6', 'request_id': 'req-562929ab-e9ba-463e-b252-cedfb3dbbfc9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.726820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671f6acbee494c978a1bab27b06598a6', 'request_id': 'req-562929ab-e9ba-463e-b252-cedfb3dbbfc9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.732999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671f6acbee494c978a1bab27b06598a6', 'request_id': 'req-562929ab-e9ba-463e-b252-cedfb3dbbfc9'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.747172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] The token cache is being invalidate because project 671f6acbee494c978a1bab27b06598a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:49.747462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.748113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.749677 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1342/2689] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:49 2022] DELETE /identity/v3/projects/671f6acbee494c978a1bab27b06598a6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:49.754001 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:49.763813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206f82d-ec94-4f12-b73f-85abb202046a service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:49.821115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:49.822025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:49.822289 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:49.822548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] PATH_INFO: `/v3/users/c95826ddf4f84ffb8e91e5a904ec6d1f` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:49.823804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c95826ddf4f84ffb8e91e5a904ec6d1f)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:49.825804 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:49.876634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c95826ddf4f84ffb8e91e5a904ec6d1f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.877359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c95826ddf4f84ffb8e91e5a904ec6d1f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.877961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c95826ddf4f84ffb8e91e5a904ec6d1f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.887067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c95826ddf4f84ffb8e91e5a904ec6d1f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.895292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:49.902007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c95826ddf4f84ffb8e91e5a904ec6d1f'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:49.915552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1348/2690] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:49 2022] DELETE /identity/v3/users/c95826ddf4f84ffb8e91e5a904ec6d1f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:08:50.193436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:50.195399 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:50.209049 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562929ab-e9ba-463e-b252-cedfb3dbbfc9 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:50.267463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:50.268354 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:50.268629 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:50.268897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] PATH_INFO: `/v3/projects/09bc8d1ac1b84b6a83aea7573bf8d0b3` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:50.270216 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09bc8d1ac1b84b6a83aea7573bf8d0b3)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:50.272703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:50.322468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09bc8d1ac1b84b6a83aea7573bf8d0b3', 'request_id': 'req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.328072 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09bc8d1ac1b84b6a83aea7573bf8d0b3', 'request_id': 'req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.334636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09bc8d1ac1b84b6a83aea7573bf8d0b3', 'request_id': 'req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.348423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] The token cache is being invalidate because project 09bc8d1ac1b84b6a83aea7573bf8d0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:50.348708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.349329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.351089 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1343/2691] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:50 2022] DELETE /identity/v3/projects/09bc8d1ac1b84b6a83aea7573bf8d0b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:50.355136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:50.356668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:50.369486 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c1211-3b5b-4f21-858f-efcd46ba2ae2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:50.428515 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:50.429406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:50.429685 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:50.429961 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] PATH_INFO: `/v3/users/6504083c89a747698a0a2bf9feeececd` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:50.431340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6504083c89a747698a0a2bf9feeececd)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:50.433679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:50.483405 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6504083c89a747698a0a2bf9feeececd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.484035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6504083c89a747698a0a2bf9feeececd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.484661 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6504083c89a747698a0a2bf9feeececd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.493360 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6504083c89a747698a0a2bf9feeececd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.505044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a7694dcb-f620-4d8d-b1c9-b0f0dc2cbac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6504083c89a747698a0a2bf9feeececd'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.518788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1349/2692] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:08:50 2022] DELETE /identity/v3/users/6504083c89a747698a0a2bf9feeececd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:50.811938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:50.813983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:50.830192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c5b69-2eef-4128-bf05-2cafe751cc1e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:50.882291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:50.901646 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:50.902730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:50.903171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:50.903553 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] PATH_INFO: `/v3/projects/85c38ab29f7a4f668648eb9de0f6f724` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:50.904967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c38ab29f7a4f668648eb9de0f6f724)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:50.908145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:50.965872 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c38ab29f7a4f668648eb9de0f6f724', 'request_id': 'req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.973466 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c38ab29f7a4f668648eb9de0f6f724', 'request_id': 'req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.980339 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c38ab29f7a4f668648eb9de0f6f724', 'request_id': 'req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:50.999830 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] The token cache is being invalidate because project 85c38ab29f7a4f668648eb9de0f6f724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:08:51.000464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:51.001499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-acd0aed4-c2c6-4c7c-b883-85c8ad870ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:08:51.004318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1344/2693] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:08:50 2022] DELETE /identity/v3/projects/85c38ab29f7a4f668648eb9de0f6f724 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:51.012813 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.013254 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.013596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.033186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:51.034528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:51.088997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:51.100086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1350/2694] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:51.110187 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.110473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.110742 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.133855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:51.135058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:51.144421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1345/2695] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:51.149630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.165075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef70a6-79d8-41cc-995d-6717fafd5d73 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.174272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.175071 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.175378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.175710 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.176761 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:08:51.177070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.179511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.192539 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1351/2696] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:08:51.197746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.207451 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed648022-1b81-4237-b265-aa1b8d879436 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.213551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.214210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.214511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.214771 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.215470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.217438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.246902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1346/2697] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:51 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:51.251186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.261887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc9220-568a-4173-b488-d4cb35deb7f3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.321073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.321751 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.322030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.322302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.322999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.325102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.348863 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1352/2698] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:51 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:51.353586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.368475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e6dc1-f1d9-460d-bf8f-599f7ad03c8e admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.373492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.374148 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.374486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.375248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.375391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.377100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.389231 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1347/2699] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:51.393156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.404596 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb27c9-a507-4580-94f3-70fd3a6d9231 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.410394 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaf698-1f2b-4e9b-8820-e0a173a944ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.411086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaf698-1f2b-4e9b-8820-e0a173a944ff admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.411340 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaf698-1f2b-4e9b-8820-e0a173a944ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.411613 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaf698-1f2b-4e9b-8820-e0a173a944ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.412424 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaf698-1f2b-4e9b-8820-e0a173a944ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.414075 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaf698-1f2b-4e9b-8820-e0a173a944ff admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.425125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1353/2700] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:51.429615 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.440251 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cf2b6-f183-49de-af6c-6c5af4357b3d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.444967 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.445857 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.446132 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.446402 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] PATH_INFO: `/v3/projects/e3312f992df04304a0b98e9cf523b7ab/users/e01140fbd03f41acaf8d2f3e5ada5384/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.461024 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3312f992df04304a0b98e9cf523b7ab, user_id=e01140fbd03f41acaf8d2f3e5ada5384, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.463684 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.481081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1348/2701] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:51 2022] PUT /identity/v3/projects/e3312f992df04304a0b98e9cf523b7ab/users/e01140fbd03f41acaf8d2f3e5ada5384/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:51.488029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac433a-4961-4037-bc74-f1fc2e388b09 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.488429 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac433a-4961-4037-bc74-f1fc2e388b09 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.488801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac433a-4961-4037-bc74-f1fc2e388b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.510662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b4ac433a-4961-4037-bc74-f1fc2e388b09 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:51.512959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b4ac433a-4961-4037-bc74-f1fc2e388b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:51.611324 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1354/2702] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:08:51.619314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:51.621272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:51.689613 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c1f0-57b5-4c7d-8cbf-ed2d7c4dc925 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:51.694220 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:51.694873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:51.695152 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:51.695427 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:51.706669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:51.708773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:51.719160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1349/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:52.403331 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.403826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.403954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.431796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] MFA Rules not processed for user `e01140fbd03f41acaf8d2f3e5ada5384`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:52.433384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:52.502637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:52.513959 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1355/2704] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:52.518573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:52.528417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d3184-ca5e-4af9-b77e-64590d66379d service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:52.532985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:52.533637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.533916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.534197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.534883 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:52.536824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:52.545560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:08:52.569449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1350/2705] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:08:52 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:08:52.570701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:52.582082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508db7-0d73-451b-8cea-54b7c43613cb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:52.641282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:52.641971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.642229 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.642508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.643188 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:52.645173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:52.667611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1356/2706] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:08:52 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:08:52.671841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:52.684244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f795-f67d-4dd8-af59-31d068cbefcf admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:52.690204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:52.690928 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.691209 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.691481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.692265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:52.693949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:52.706292 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1351/2707] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:52.711370 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:52.723481 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874ba65-6a09-4a67-ba6d-48eba95f2464 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:52.729867 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:52.730870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.731159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.731425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] PATH_INFO: `/v3/projects/24413ae9741242c5aa2af507f2edc246/users/0e86845069534b1d93e0d7f251177b0f/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.749097 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24413ae9741242c5aa2af507f2edc246, user_id=0e86845069534b1d93e0d7f251177b0f, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:52.751794 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:52.768108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1357/2708] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:52 2022] PUT /identity/v3/projects/24413ae9741242c5aa2af507f2edc246/users/0e86845069534b1d93e0d7f251177b0f/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:52.772438 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:52.783433 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8eeb-603b-4726-a8f9-1d495f819808 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:52.851076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a97f39-f77b-4aaa-b3d2-e402d130d477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:52.851748 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a97f39-f77b-4aaa-b3d2-e402d130d477 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.852059 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a97f39-f77b-4aaa-b3d2-e402d130d477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.852336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a97f39-f77b-4aaa-b3d2-e402d130d477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.853068 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a97f39-f77b-4aaa-b3d2-e402d130d477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:52.854668 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a97f39-f77b-4aaa-b3d2-e402d130d477 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:52.869081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1352/2709] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:08:52.870466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:52.881934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff770f-995f-46ae-a821-5157dcd3f4c2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:52.886592 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:52.887523 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:52.887815 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:52.888115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] PATH_INFO: `/v3/projects/24413ae9741242c5aa2af507f2edc246/users/0e86845069534b1d93e0d7f251177b0f/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:52.890182 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24413ae9741242c5aa2af507f2edc246, user_id=0e86845069534b1d93e0d7f251177b0f, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:52.892906 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:52.905882 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1358/2710] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:08:52 2022] PUT /identity/v3/projects/24413ae9741242c5aa2af507f2edc246/users/0e86845069534b1d93e0d7f251177b0f/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:08:53.946262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:53.946783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:53.946890 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:53.977264 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] MFA Rules not processed for user `0e86845069534b1d93e0d7f251177b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:08:53.978682 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:54.056293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1353/2711] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:08:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:08:54.068423 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:54.070692 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:54.134417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d35a7-609f-4729-992e-77701e31e85d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:54.138117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:54.138783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:54.139069 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:54.139350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:54.200013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:54.201617 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:54.211022 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1359/2712] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:08:55.579135 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:08:55.580680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:08:55.592938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19bbac-05c3-4422-9cd6-a379043076d1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:08:55.596672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:08:55.597363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:08:55.597650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:08:55.597931 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:08:55.609984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:08:55.611468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:08:55.621661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1354/2713] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:08:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:09:04.206624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:INSERT=5 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:05.617605 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:09.040698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:09.042145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:09.110031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:09.123799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954b698-d77d-4e71-9f23-fee3477286ab service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:09.128873 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:09.129531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:09.129809 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:09.130081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:09.131849 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:09.143701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:09.145307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:09.155311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1360/2714] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:09:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:09:09.179770 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a579f-f8be-45df-aeb0-c5a1b2146e6d service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:09.185211 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:09.186267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:09.186552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:09.186870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] PATH_INFO: `/v3/users/e01140fbd03f41acaf8d2f3e5ada5384` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:09.188122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e01140fbd03f41acaf8d2f3e5ada5384)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:09.190064 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:09.245256 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e01140fbd03f41acaf8d2f3e5ada5384'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.245909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e01140fbd03f41acaf8d2f3e5ada5384'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.254485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e01140fbd03f41acaf8d2f3e5ada5384'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.255090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e01140fbd03f41acaf8d2f3e5ada5384'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.267577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e01140fbd03f41acaf8d2f3e5ada5384'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.279881 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1355/2715] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:09:09 2022] DELETE /identity/v3/users/e01140fbd03f41acaf8d2f3e5ada5384 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:09.475029 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:09.489187 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513752-10e9-4639-9de7-d4080d1571d1 service placement] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:09.557836 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:09.558975 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:09.559312 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:09.559703 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] PATH_INFO: `/v3/projects/e3312f992df04304a0b98e9cf523b7ab` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:09.561372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3312f992df04304a0b98e9cf523b7ab)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:09.563295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:09.620231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3312f992df04304a0b98e9cf523b7ab', 'request_id': 'req-91ad274f-46db-4b94-91a9-50aa2abaecd2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.626978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3312f992df04304a0b98e9cf523b7ab', 'request_id': 'req-91ad274f-46db-4b94-91a9-50aa2abaecd2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.636325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3312f992df04304a0b98e9cf523b7ab', 'request_id': 'req-91ad274f-46db-4b94-91a9-50aa2abaecd2'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.648179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] The token cache is being invalidate because project e3312f992df04304a0b98e9cf523b7ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:09:09.648463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.649235 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.650835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1361/2716] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:09:09 2022] DELETE /identity/v3/projects/e3312f992df04304a0b98e9cf523b7ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:09.655286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:09.656828 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:09.669765 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9eba0-1ece-4043-af19-2c4e3385026b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:09.729409 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:09.730301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:09.730662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:09.731061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] PATH_INFO: `/v3/users/0e86845069534b1d93e0d7f251177b0f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:09.732392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e86845069534b1d93e0d7f251177b0f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:09.734417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:09.756577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:09.784357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e86845069534b1d93e0d7f251177b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.785013 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e86845069534b1d93e0d7f251177b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.793538 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e86845069534b1d93e0d7f251177b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.794188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e86845069534b1d93e0d7f251177b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.806321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-b147854d-27eb-4a3e-840a-f73cf4659232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e86845069534b1d93e0d7f251177b0f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:09.818787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1356/2717] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:09:09 2022] DELETE /identity/v3/users/0e86845069534b1d93e0d7f251177b0f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:09.994322 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:09.995943 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:10.010631 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad274f-46db-4b94-91a9-50aa2abaecd2 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.031408 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:10.078325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.079202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.079482 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.079756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] PATH_INFO: `/v3/projects/24413ae9741242c5aa2af507f2edc246` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.081044 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24413ae9741242c5aa2af507f2edc246)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.083002 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.138296 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24413ae9741242c5aa2af507f2edc246', 'request_id': 'req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:10.144799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24413ae9741242c5aa2af507f2edc246', 'request_id': 'req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:10.155737 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24413ae9741242c5aa2af507f2edc246', 'request_id': 'req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:10.168171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] The token cache is being invalidate because project 24413ae9741242c5aa2af507f2edc246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:09:10.168456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:10.169139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-35d88e91-1f2d-4f8f-82b2-2afbaa92c750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:10.171177 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1362/2718] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:09:09 2022] DELETE /identity/v3/projects/24413ae9741242c5aa2af507f2edc246 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:10.176408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.176905 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.177018 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.198921 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:10.200159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:10.262429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1357/2719] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:10.270657 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.271020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.271359 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.294643 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:10.296179 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:10.305927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1363/2720] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:10.311913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.326006 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944e985-34c9-4f04-9670-9d34d7e5f0e7 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.337616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.338674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.338973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.339238 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.341841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:09:10.341841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.342087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.354719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1358/2721] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:09:10.360369 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.374884 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9e76f-f99b-4597-a7a2-00637cd04142 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.381171 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.381829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.382133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.382406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.383102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.385045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.415434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1364/2722] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:09:10 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:09:10.421525 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.433576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8a82e-1765-4d1c-ae79-88ce9ccc984d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.490521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.491179 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.491457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.491731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.492450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.494373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.520558 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1359/2723] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:09:10 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:09:10.524914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.535937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889bc4-8f97-41dd-bc29-7dce61be6517 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.541477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.542130 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.542411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.542675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.543401 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.545030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.555931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1365/2724] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:09:10.560449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.570655 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1782a4-1d6f-435b-b90d-5b772237c251 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.575913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180566e-2344-41db-9116-2752ea7a5c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.576567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180566e-2344-41db-9116-2752ea7a5c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.576912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180566e-2344-41db-9116-2752ea7a5c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.577234 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180566e-2344-41db-9116-2752ea7a5c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.577981 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180566e-2344-41db-9116-2752ea7a5c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.579581 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180566e-2344-41db-9116-2752ea7a5c88 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.589887 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1360/2725] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:09:10.594433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.605247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4f31f-c2b4-4902-a2b1-107e39b6a114 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.611072 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.611980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.612255 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.612525 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] PATH_INFO: `/v3/projects/0e59985cc727498f88959366da26b291/users/5c3447c7323f4c569b960d479c55030f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.630441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e59985cc727498f88959366da26b291, user_id=5c3447c7323f4c569b960d479c55030f, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.633153 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.647668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1366/2726] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:09:10 2022] PUT /identity/v3/projects/0e59985cc727498f88959366da26b291/users/5c3447c7323f4c569b960d479c55030f/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:10.658499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.658788 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.659066 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.666567 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:10.691272 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] MFA Rules not processed for user `5c3447c7323f4c569b960d479c55030f`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:10.692517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:10.761347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1361/2727] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:09:10.773978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:10.775739 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:10.840914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:10.848366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531397b1-3c55-4c45-98e7-496d4bc3fef4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:10.854534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051eed10-4c3d-41b8-a021-5ac47350b40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:10.855542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051eed10-4c3d-41b8-a021-5ac47350b40f service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:10.856232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051eed10-4c3d-41b8-a021-5ac47350b40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:10.856605 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051eed10-4c3d-41b8-a021-5ac47350b40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:10.866698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051eed10-4c3d-41b8-a021-5ac47350b40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:10.868212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051eed10-4c3d-41b8-a021-5ac47350b40f service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:10.877971 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1367/2728] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:09:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:09:13.447362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:13.528915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe5279-aa0e-45bf-aa07-e2aaafcdf546 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:13.534733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:13.535611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:13.535904 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:13.536201 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] PATH_INFO: `/v3/users/5c3447c7323f4c569b960d479c55030f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:13.537536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3447c7323f4c569b960d479c55030f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:13.539468 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:13.595382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3447c7323f4c569b960d479c55030f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:13.596092 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3447c7323f4c569b960d479c55030f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:13.605569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3447c7323f4c569b960d479c55030f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:13.606450 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3447c7323f4c569b960d479c55030f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:13.620307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3447c7323f4c569b960d479c55030f'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:13.632806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1362/2729] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:09:13 2022] DELETE /identity/v3/users/5c3447c7323f4c569b960d479c55030f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:13.637662 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:13.638146 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:13.638261 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:13.655133 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:13.656372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:13.724304 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1368/2730] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:13.734373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:13.735960 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:13.751947 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:13.805893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67421102-5a05-48f2-9c8d-da1c00e27780 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:13.810455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218a0d1-6e70-4764-b2a9-fda8eb570c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:13.811100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218a0d1-6e70-4764-b2a9-fda8eb570c50 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:13.811377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218a0d1-6e70-4764-b2a9-fda8eb570c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:13.811653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218a0d1-6e70-4764-b2a9-fda8eb570c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:13.821412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218a0d1-6e70-4764-b2a9-fda8eb570c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:13.822852 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218a0d1-6e70-4764-b2a9-fda8eb570c50 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:13.832783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1363/2731] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:09:14.109816 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.111376 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:14.124419 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5628c1-59b8-4db1-9188-200fcd35802f None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.130113 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.131024 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.131298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.131564 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] PATH_INFO: `/v3/projects/0e59985cc727498f88959366da26b291` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.132923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e59985cc727498f88959366da26b291)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.134879 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.194345 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e59985cc727498f88959366da26b291', 'request_id': 'req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:14.201300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e59985cc727498f88959366da26b291', 'request_id': 'req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:14.211618 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e59985cc727498f88959366da26b291', 'request_id': 'req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:14.226652 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] The token cache is being invalidate because project 0e59985cc727498f88959366da26b291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:09:14.226940 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:14.227709 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1fa1d381-fef9-4edf-9e3a-36f3af32c8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:09:14.229649 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1369/2732] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:09:14 2022] DELETE /identity/v3/projects/0e59985cc727498f88959366da26b291 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:14.236454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.236740 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.237015 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.256648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:14.258639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:14.321656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1364/2733] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:14.387347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.387635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.388116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.412398 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:14.413625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:14.422885 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1370/2734] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:14.427508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.437903 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177fcb0-ae4e-487b-849e-49fd51bca084 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.443743 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.444432 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.444714 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.444987 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.445719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:09:14.446026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.447637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.460185 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1365/2735] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:09:14.465343 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.480046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71143b8f-50cd-4485-bbab-89720f991872 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.488336 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.488983 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.489248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.489516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.490206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.492122 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.524109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1371/2736] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:09:14 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:09:14.530560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.547273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd59304-28af-49bb-94f9-2531badd3b28 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.607446 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.608160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.608442 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.608716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.609412 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.611373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.634204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1366/2737] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:09:14 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:09:14.638650 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.649232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817e2e9-5a07-4f2f-983b-856e67359656 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.653868 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.654530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.654808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.655076 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.655829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.657461 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.668425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1372/2738] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:09:14.672524 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.686625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244b5cf-e784-48d4-b598-573af2c8c507 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.693145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d656ef8-eb84-4d8b-9098-21deeb036b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.693814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d656ef8-eb84-4d8b-9098-21deeb036b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.694096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d656ef8-eb84-4d8b-9098-21deeb036b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.694367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d656ef8-eb84-4d8b-9098-21deeb036b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.695101 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d656ef8-eb84-4d8b-9098-21deeb036b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.696759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d656ef8-eb84-4d8b-9098-21deeb036b05 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.708193 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1367/2739] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:09:14.712905 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.727020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefe8f-3ce5-4c08-8af6-b6220f76cd84 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.733091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.734176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.734507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.734848 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] PATH_INFO: `/v3/projects/84c9f15ea7b84f54a6cfa004bd02de6e/users/fbc71c372e584ddf91290eefb53a973a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.754286 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c9f15ea7b84f54a6cfa004bd02de6e, user_id=fbc71c372e584ddf91290eefb53a973a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.757406 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.774866 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1373/2740] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:09:14 2022] PUT /identity/v3/projects/84c9f15ea7b84f54a6cfa004bd02de6e/users/fbc71c372e584ddf91290eefb53a973a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:09:14.781552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d2325-9bd8-46da-8430-7c3c81d676ad None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.781925 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d2325-9bd8-46da-8430-7c3c81d676ad None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.782276 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d2325-9bd8-46da-8430-7c3c81d676ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.802570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d15d2325-9bd8-46da-8430-7c3c81d676ad None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:14.803799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d15d2325-9bd8-46da-8430-7c3c81d676ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:14.824985 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:14.870595 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1368/2741] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:09:14.881093 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:14.883358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:14.894646 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:14.943670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe522-ce39-4905-b588-7038dfb39df3 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:14.948141 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:14.948779 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:14.949060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:14.949302 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:14.959660 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:14.960919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:14.971030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1374/2742] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:09:19.049254 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.050481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.050481 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.082376 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] MFA Rules not processed for user `fbc71c372e584ddf91290eefb53a973a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:19.083628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:19.152050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1369/2743] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:19.156817 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:19.168805 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f0b1d-c31a-4b16-aff8-ed5be3b9d062 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:19.174125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:19.175064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.175476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.175752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.177237 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:19.178766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:19.220698 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1375/2744] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:09:19 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:09:19.224864 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:19.235732 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1852-7f4a-40f0-8c41-cd9e251da7df None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:19.301949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:19.302680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.302966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.303273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.304047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:19.306050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:19.330263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1370/2745] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:09:19 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:09:19.334732 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:19.346025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9305-efe6-44e7-9ee7-87448d7bd4f4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:19.351207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:19.351966 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.352386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.352503 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.353232 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:19.354921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:19.370112 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1376/2746] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:09:19.371362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:19.381913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c244e8a-1f36-4915-bc0a-12c0de1bfe16 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:19.386665 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:19.387566 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.387842 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.388177 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] PATH_INFO: `/v3/projects/bfa9157dcdfd4183a53edc3cf25566e7/users/262491bdaffd47adaa2d174b5160e0ac/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.404902 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa9157dcdfd4183a53edc3cf25566e7, user_id=262491bdaffd47adaa2d174b5160e0ac, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:19.407589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:19.423441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1371/2747] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:09:19 2022] PUT /identity/v3/projects/bfa9157dcdfd4183a53edc3cf25566e7/users/262491bdaffd47adaa2d174b5160e0ac/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:19.428183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:19.439673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6532b6-29a8-401c-ab43-5872c18ed137 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:19.503725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e0495-6603-462c-b7b8-6fbc924a7100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:19.504526 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e0495-6603-462c-b7b8-6fbc924a7100 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.504875 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e0495-6603-462c-b7b8-6fbc924a7100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.505226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e0495-6603-462c-b7b8-6fbc924a7100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.506035 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e0495-6603-462c-b7b8-6fbc924a7100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:19.507714 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e0495-6603-462c-b7b8-6fbc924a7100 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:19.519333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1377/2748] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:09:19.523814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:19.535417 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e0aa0-6cbd-439b-8b96-97c2a27c64a5 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:19.541133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:19.542054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:19.542336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:19.542616 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] PATH_INFO: `/v3/projects/bfa9157dcdfd4183a53edc3cf25566e7/users/262491bdaffd47adaa2d174b5160e0ac/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:19.545061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa9157dcdfd4183a53edc3cf25566e7, user_id=262491bdaffd47adaa2d174b5160e0ac, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:19.547794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:19.566462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1372/2749] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:09:19 2022] PUT /identity/v3/projects/bfa9157dcdfd4183a53edc3cf25566e7/users/262491bdaffd47adaa2d174b5160e0ac/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:09:23.999417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a748da1-8a41-4cb4-8bef-88453e5a3ae4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:23.999976 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a748da1-8a41-4cb4-8bef-88453e5a3ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:24.000231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a748da1-8a41-4cb4-8bef-88453e5a3ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:24.028434 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-6a748da1-8a41-4cb4-8bef-88453e5a3ae4 None None] MFA Rules not processed for user `262491bdaffd47adaa2d174b5160e0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:24.029696 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6a748da1-8a41-4cb4-8bef-88453e5a3ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:24.101185 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:24.106677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1378/2750] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:24.124454 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:24.127281 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:24.134155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:24.210662 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b909fc6-747e-4f29-8dee-64a1279f7f98 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:24.217724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3f18b-b5be-40a8-9210-f63ac32810a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:24.217975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3f18b-b5be-40a8-9210-f63ac32810a7 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:24.218093 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3f18b-b5be-40a8-9210-f63ac32810a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:24.218333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3f18b-b5be-40a8-9210-f63ac32810a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:24.280416 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3f18b-b5be-40a8-9210-f63ac32810a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:24.282062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3f18b-b5be-40a8-9210-f63ac32810a7 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:24.297722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1373/2751] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:09:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:09:25.456855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:25.457282 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:25.457391 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:25.477048 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] MFA Rules not processed for user `fbc71c372e584ddf91290eefb53a973a`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:25.478330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:25.485780 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1379/2752] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:34.288760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=54 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:35.470808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:36.029611 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:36.030110 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:36.030239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:36.047898 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] MFA Rules not processed for user `3593c14c5890431d8c4a986132a659b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:09:36.049113 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:09:36.104837 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1374/2753] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:09:46.098091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=25 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:09:56.723088 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:09:56.797129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c291b-4745-4049-ae70-7e65ae60d932 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:09:56.801413 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:09:56.802004 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:09:56.802265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:09:56.802567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:09:56.812945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:09:56.814387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:09:56.824512 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1380/2754] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:09:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:10:06.819926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=26 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:10:24.570497 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:24.572820 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:24.640003 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5583281-3b66-40a2-aec8-f7fc78659f69 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:24.644510 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:24.645155 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:24.645431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:24.645701 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:24.647675 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:24.661772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:24.663199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f31dc-4de9-47c3-9ef2-7a85f4466027 service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:24.674818 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1375/2755] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:10:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:10:32.874667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:32.876246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:32.889736 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2374-1db8-4b4c-91e4-b295b4a81339 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:32.894136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:32.894764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:32.895041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:32.895415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:32.897435 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:32.911339 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:32.912801 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:32.924118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1381/2756] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:10:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:10:34.669265 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=26 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:10:41.832367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:41.833007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:41.833007 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:41.852805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] MFA Rules not processed for user `ead8ee09f8514e75882c877042764a92`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:10:41.854020 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:41.913098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1376/2757] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:10:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:10:42.918830 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=6 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:10:46.708548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:46.718957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9209f-1348-4cfd-bbe1-05c8557de243 service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:46.722639 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:46.723292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:46.723569 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:46.723839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:46.725795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:46.737778 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:46.739743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:46.750545 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1382/2758] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:10:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:10:50.964441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:50.982313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe00f7e-6eb5-4988-9fd4-a3cac3bbe843 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:50.989382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:50.990037 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:50.990314 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:50.990584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:50.992861 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:51.004730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:51.006342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:51.016312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1377/2759] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:10:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:10:56.709973 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:56.711495 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:56.784851 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c52c7-a01b-4c6c-b18a-b92d5c2bbe94 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:56.789320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:56.790207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:56.790480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:56.790749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] PATH_INFO: `/v3/users/3593c14c5890431d8c4a986132a659b4` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:56.791985 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3593c14c5890431d8c4a986132a659b4)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:56.794151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:56.843381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3593c14c5890431d8c4a986132a659b4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:56.844059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3593c14c5890431d8c4a986132a659b4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:56.844640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3593c14c5890431d8c4a986132a659b4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:56.853349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3593c14c5890431d8c4a986132a659b4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:56.865629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-90976aa8-ed92-4fd7-8eb9-16701169ab3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3593c14c5890431d8c4a986132a659b4'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:56.878137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1383/2760] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:10:56 2022] DELETE /identity/v3/users/3593c14c5890431d8c4a986132a659b4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:10:57.061520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.073387 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fd548-52f9-4001-afef-64221530b8e0 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.132398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.133180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.133460 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.133720 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] PATH_INFO: `/v3/projects/315760a46046430780d7d4741cd8dc7f` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.134929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315760a46046430780d7d4741cd8dc7f)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.136873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.187506 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315760a46046430780d7d4741cd8dc7f', 'request_id': 'req-bb021f68-a505-4eae-b982-6e5259753d25'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:57.194121 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315760a46046430780d7d4741cd8dc7f', 'request_id': 'req-bb021f68-a505-4eae-b982-6e5259753d25'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:57.200929 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315760a46046430780d7d4741cd8dc7f', 'request_id': 'req-bb021f68-a505-4eae-b982-6e5259753d25'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:57.216731 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] The token cache is being invalidate because project 315760a46046430780d7d4741cd8dc7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:10:57.216999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:57.218240 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-bb021f68-a505-4eae-b982-6e5259753d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:10:57.219656 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1378/2761] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:10:57 2022] DELETE /identity/v3/projects/315760a46046430780d7d4741cd8dc7f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:10:57.225159 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.225556 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.225898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.244620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:10:57.245977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:57.310559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1384/2762] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:10:57.319863 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.320383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.320882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.345410 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:10:57.346667 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:57.362782 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1379/2763] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:10:57.367101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.377783 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8d969-edae-4a6f-b128-6d57c7f0e709 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.382890 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.383954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.384590 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.385091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.386460 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:10:57.387030 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.390189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.392855 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:10:57.406420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1385/2764] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:10:57.410576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.420657 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3d3ac-9a90-4406-90dd-1be57f948689 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.425543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.426235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.426520 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.426795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.427504 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.429846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.443811 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:10:57.459428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1380/2765] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:10:57 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:10:57.463764 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.477292 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111a466-d621-4b85-b4ec-94b052157d74 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.537965 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.538635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.538916 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.539186 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.539918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.541839 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.564681 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1386/2766] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:10:57 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:10:57.569289 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.579559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca169c-7857-462e-8215-fe27f79bec1a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.584162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.584809 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.585091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.585369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.586107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.587718 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.598162 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1381/2767] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:10:57.602077 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.612530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a28599-2517-44bf-87f0-8497a253134c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.618334 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0befe-4d15-488c-bbfb-a031f0c98fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.618997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0befe-4d15-488c-bbfb-a031f0c98fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.619273 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0befe-4d15-488c-bbfb-a031f0c98fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.619546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0befe-4d15-488c-bbfb-a031f0c98fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.620320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0befe-4d15-488c-bbfb-a031f0c98fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.621926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0befe-4d15-488c-bbfb-a031f0c98fc4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.632380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1387/2768] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:10:57.636535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.646897 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70840-3475-4981-89ef-4f31ac7d8ca2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.651461 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.652381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.653472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.653823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] PATH_INFO: `/v3/projects/c7304ecf653042e4914899798cfcdee5/users/1de745cdb7ab41eca04241cb49aff875/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.669795 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7304ecf653042e4914899798cfcdee5, user_id=1de745cdb7ab41eca04241cb49aff875, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.672484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.689854 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1382/2769] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:10:57 2022] PUT /identity/v3/projects/c7304ecf653042e4914899798cfcdee5/users/1de745cdb7ab41eca04241cb49aff875/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:10:57.703624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.703939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.704218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.733311 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] MFA Rules not processed for user `1de745cdb7ab41eca04241cb49aff875`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:10:57.735320 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:57.808918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1388/2770] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:10:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:10:57.821846 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:10:57.823719 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:10:57.899753 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b966-f825-4fe8-9c13-f8a1428260e3 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:10:57.906188 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2af803-e075-4551-b2eb-8090eec1c5df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:10:57.906841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2af803-e075-4551-b2eb-8090eec1c5df service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:10:57.907120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2af803-e075-4551-b2eb-8090eec1c5df service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:10:57.907391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2af803-e075-4551-b2eb-8090eec1c5df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:10:57.915144 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2af803-e075-4551-b2eb-8090eec1c5df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:10:57.916772 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2af803-e075-4551-b2eb-8090eec1c5df service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:10:57.926543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1383/2771] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:10:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:05.290939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:05.355527 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645a844-97bc-4265-b5c3-ca0e66a69669 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:05.360865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:05.361749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:05.362027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:05.362295 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] PATH_INFO: `/v3/users/1de745cdb7ab41eca04241cb49aff875` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:05.363535 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de745cdb7ab41eca04241cb49aff875)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:05.365491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:05.367802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:05.415995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de745cdb7ab41eca04241cb49aff875'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:05.416571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de745cdb7ab41eca04241cb49aff875'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:05.417164 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de745cdb7ab41eca04241cb49aff875'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:05.425964 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de745cdb7ab41eca04241cb49aff875'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:05.439361 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de745cdb7ab41eca04241cb49aff875'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:05.453103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1389/2772] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:05 2022] DELETE /identity/v3/users/1de745cdb7ab41eca04241cb49aff875 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:05.458222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:05.458723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:05.458836 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:05.477192 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:05.478388 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:05.541783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1384/2773] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:11:05.551325 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:05.552811 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:05.611476 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eb870-a9c7-46d0-928c-2f5dc173a8d4 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:05.615796 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360e5b-f91b-40cb-aebc-5922877e57e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:05.616480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360e5b-f91b-40cb-aebc-5922877e57e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:05.616756 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360e5b-f91b-40cb-aebc-5922877e57e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:05.617025 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360e5b-f91b-40cb-aebc-5922877e57e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:05.627637 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360e5b-f91b-40cb-aebc-5922877e57e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:05.629118 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360e5b-f91b-40cb-aebc-5922877e57e2 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:05.638752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1390/2774] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:05.924131 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:05.925570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:05.938411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259fa4c-6890-4d8b-bab1-db63c3acf921 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:05.943900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:05.944806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:05.945040 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:05.945351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] PATH_INFO: `/v3/projects/c7304ecf653042e4914899798cfcdee5` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:05.946563 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7304ecf653042e4914899798cfcdee5)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:05.948604 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:05.992556 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:06.000396 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7304ecf653042e4914899798cfcdee5', 'request_id': 'req-099699c3-2de4-40ad-bf27-426db59c6641'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:06.006527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7304ecf653042e4914899798cfcdee5', 'request_id': 'req-099699c3-2de4-40ad-bf27-426db59c6641'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:06.012997 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7304ecf653042e4914899798cfcdee5', 'request_id': 'req-099699c3-2de4-40ad-bf27-426db59c6641'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:06.027441 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] The token cache is being invalidate because project c7304ecf653042e4914899798cfcdee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:06.027730 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:06.028457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-099699c3-2de4-40ad-bf27-426db59c6641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:06.030243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1385/2775] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:05 2022] DELETE /identity/v3/projects/c7304ecf653042e4914899798cfcdee5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:06.035600 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.035914 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.036194 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.055301 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:06.056560 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:06.120950 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1391/2776] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:06.129128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.129455 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.129710 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.149922 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:06.151120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:06.160702 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1386/2777] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:06.165533 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.178640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe909f-f09e-4089-88b7-e1b925260b63 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.184246 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.184937 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.185218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.185528 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] PATH_INFO: `/v3/domains` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.186300 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:11:06.186827 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.188521 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.201095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1392/2778] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:11:06.205871 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.216041 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21305baf-13aa-403f-9bf5-590d67122801 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.221190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.221838 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.222122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.222392 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.223091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.225177 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.254723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1387/2779] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:06.259381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.273441 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a73290-7647-4d0c-bcde-b6773fcc9618 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.283135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:06.336091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.336825 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.337109 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.337380 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.338082 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.340032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.363422 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1393/2780] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:11:06.369499 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.382164 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6df34-ddac-45b1-98bf-11d171ea6ada admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.387654 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.388424 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.388705 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.388979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.389936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.391579 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.406437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1388/2781] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:06.410674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.421397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c7c34-2658-4b5e-86d7-8ef5efd545fa admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.427274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32301d5-1f34-472a-b912-7ee60156b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.428018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32301d5-1f34-472a-b912-7ee60156b6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.428293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32301d5-1f34-472a-b912-7ee60156b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.428647 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32301d5-1f34-472a-b912-7ee60156b6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.429847 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32301d5-1f34-472a-b912-7ee60156b6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.431442 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32301d5-1f34-472a-b912-7ee60156b6c7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.442614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1394/2782] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:11:06.446870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.458191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af990ae-0089-46e7-8e78-aa91a77e0197 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.463940 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.464948 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.465233 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.465517 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] PATH_INFO: `/v3/projects/ace662a2eced42f7913309facb1eb476/users/e18aa2bac65f4334998934756a654c09/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.483280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace662a2eced42f7913309facb1eb476, user_id=e18aa2bac65f4334998934756a654c09, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.486589 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.505355 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1389/2783] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:06 2022] PUT /identity/v3/projects/ace662a2eced42f7913309facb1eb476/users/e18aa2bac65f4334998934756a654c09/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:06.519593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.519898 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.520257 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.547787 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] MFA Rules not processed for user `e18aa2bac65f4334998934756a654c09`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:06.549073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:06.621167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1395/2784] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:06.626511 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.691576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb1d0a-f92c-4e76-9ed4-3a4742c59ca8 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.697232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.697936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.698257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.698562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.699369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.701309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.728637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1390/2785] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:06.732725 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.744018 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd16ef-031a-439f-83f5-eccfc984b8ac None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.802193 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.802876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.803166 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.803440 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] PATH_INFO: `/v3/users` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.804511 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.806494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.831028 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1396/2786] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:06 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:06.837570 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.850369 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a861ef-efd7-4559-9709-93fb0c902333 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.855851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.856552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.856896 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.857171 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.857916 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.859612 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.870912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1391/2787] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:06.875176 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.886285 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9dd53-a0f5-4858-8661-0f845d7f0321 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.891610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:06.892576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:06.892854 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:06.893126 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] PATH_INFO: `/v3/projects/e0656eddda3241b8955665004cd96e00/users/8f5f81411541474389074f05f03d03bc/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:06.896213 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:06.909007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0656eddda3241b8955665004cd96e00, user_id=8f5f81411541474389074f05f03d03bc, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:06.911993 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:06.926741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1397/2788] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:06 2022] PUT /identity/v3/projects/e0656eddda3241b8955665004cd96e00/users/8f5f81411541474389074f05f03d03bc/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:06.931246 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:06.941889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38ac6c-bf45-4dcc-b3db-98c2141d14fa admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:06.978794 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:07.004630 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a2522-98f2-4336-b758-5c1f3e2c963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:07.005308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a2522-98f2-4336-b758-5c1f3e2c963b admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.005626 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a2522-98f2-4336-b758-5c1f3e2c963b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.005895 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a2522-98f2-4336-b758-5c1f3e2c963b admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.006620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a2522-98f2-4336-b758-5c1f3e2c963b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:07.008262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a2522-98f2-4336-b758-5c1f3e2c963b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:07.018799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1392/2789] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:11:07.023068 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:07.034014 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5280-29b3-436f-9ae8-00aecd23de1f admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:07.039364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:07.040310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.040612 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.040840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] PATH_INFO: `/v3/projects/e0656eddda3241b8955665004cd96e00/users/8f5f81411541474389074f05f03d03bc/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.042931 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0656eddda3241b8955665004cd96e00, user_id=8f5f81411541474389074f05f03d03bc, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:07.045634 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:07.057041 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1398/2790] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:07 2022] PUT /identity/v3/projects/e0656eddda3241b8955665004cd96e00/users/8f5f81411541474389074f05f03d03bc/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:07.067470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.067759 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.068073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.095473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] MFA Rules not processed for user `8f5f81411541474389074f05f03d03bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:07.096814 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:07.178717 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1393/2791] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:07.191960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:07.194918 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:07.263492 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7de796-69db-4717-8d5c-8a5800c26e1e admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:07.267711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2499f9-99a1-4c03-b74e-2889bafed824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:07.268396 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2499f9-99a1-4c03-b74e-2889bafed824 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.268670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2499f9-99a1-4c03-b74e-2889bafed824 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.268944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2499f9-99a1-4c03-b74e-2889bafed824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.280502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2499f9-99a1-4c03-b74e-2889bafed824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:07.283463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2499f9-99a1-4c03-b74e-2889bafed824 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:07.294666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1399/2792] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:07.361274 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:07.428687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b575a-eacc-4661-af59-80ea1fcfc97b None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:07.434202 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:07.435081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.435365 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.435643 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] PATH_INFO: `/v3/users/e18aa2bac65f4334998934756a654c09` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.436891 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18aa2bac65f4334998934756a654c09)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:07.439311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:07.500827 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18aa2bac65f4334998934756a654c09'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:07.501648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18aa2bac65f4334998934756a654c09'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:07.511137 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18aa2bac65f4334998934756a654c09'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:07.511847 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18aa2bac65f4334998934756a654c09'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:07.519032 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:07.528876 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18aa2bac65f4334998934756a654c09'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:07.542153 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1394/2793] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:07 2022] DELETE /identity/v3/users/e18aa2bac65f4334998934756a654c09 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:11:07.548716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.549205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.549447 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.568769 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:07.569902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:07.637957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1400/2794] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:11:07.648551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:07.650057 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:07.712463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dc258-b456-4d2a-ac81-4f5a1dd4841a admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:07.717383 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:07.718307 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:07.718672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:07.719097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:07.728639 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:07.730106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:07.739754 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1395/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:08.027052 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:08.028832 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:08.045348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114162-9172-4371-85bf-01fde71cab53 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:08.051087 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:08.051899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.052250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.052425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] PATH_INFO: `/v3/projects/ace662a2eced42f7913309facb1eb476` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.053772 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace662a2eced42f7913309facb1eb476)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:08.056522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:08.114431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace662a2eced42f7913309facb1eb476', 'request_id': 'req-e8b73072-b8d1-424c-b4f9-8c1754811ad8'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.119728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace662a2eced42f7913309facb1eb476', 'request_id': 'req-e8b73072-b8d1-424c-b4f9-8c1754811ad8'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.128926 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace662a2eced42f7913309facb1eb476', 'request_id': 'req-e8b73072-b8d1-424c-b4f9-8c1754811ad8'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.136059 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:08.141216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] The token cache is being invalidate because project ace662a2eced42f7913309facb1eb476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:08.141519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.142379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.146149 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1401/2796] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:08 2022] DELETE /identity/v3/projects/ace662a2eced42f7913309facb1eb476 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:08.150443 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:08.160678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a898-9dcf-43ec-b650-9e7bf115567a service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:08.221635 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:08.222515 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.222793 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.223061 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] PATH_INFO: `/v3/users/8f5f81411541474389074f05f03d03bc` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.224331 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5f81411541474389074f05f03d03bc)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:08.226268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:08.288269 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5f81411541474389074f05f03d03bc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.289031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5f81411541474389074f05f03d03bc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.298226 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5f81411541474389074f05f03d03bc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.298874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5f81411541474389074f05f03d03bc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.311806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f12d3d56-7125-4556-aac0-956cd0b4f386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5f81411541474389074f05f03d03bc'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.320760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:08.325866 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1396/2797] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:08 2022] DELETE /identity/v3/users/8f5f81411541474389074f05f03d03bc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:08.611578 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:08.613116 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:08.626507 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b73072-b8d1-424c-b4f9-8c1754811ad8 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:08.685484 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:08.686375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.686665 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.686947 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] PATH_INFO: `/v3/projects/e0656eddda3241b8955665004cd96e00` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.688279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0656eddda3241b8955665004cd96e00)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:08.690230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:08.742386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0656eddda3241b8955665004cd96e00', 'request_id': 'req-15f4db8c-cc93-4902-8877-a420a1f8c037'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.748051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0656eddda3241b8955665004cd96e00', 'request_id': 'req-15f4db8c-cc93-4902-8877-a420a1f8c037'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.756970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0656eddda3241b8955665004cd96e00', 'request_id': 'req-15f4db8c-cc93-4902-8877-a420a1f8c037'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.769321 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] The token cache is being invalidate because project e0656eddda3241b8955665004cd96e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:08.769614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.770337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-15f4db8c-cc93-4902-8877-a420a1f8c037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:08.772090 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1402/2798] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:08 2022] DELETE /identity/v3/projects/e0656eddda3241b8955665004cd96e00 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:11:08.777232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.777750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.777750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.796594 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:08.798011 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:08.862213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1397/2799] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:11:08.871231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.871514 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.871789 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.890958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:08.892079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:08.900594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1403/2800] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:08.905485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:08.921372 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09545b0-f83d-4798-b29d-db709cd320e1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:08.927672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:08.928380 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.928659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.929267 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.930010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:11:08.930318 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:08.931975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:08.944290 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1398/2801] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:11:08.948607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:08.958104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ac2a5-f4be-4a57-b1c6-156924b910cb None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:08.962952 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:08.963595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:08.963892 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:08.964172 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:08.964865 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:08.966754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:08.994473 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1404/2802] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:08 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:08.998636 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.008851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13bd8-648e-4190-b654-84d8a0577e55 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.065689 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.066361 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.066633 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.066906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.067609 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.069555 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.093058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1399/2803] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:08 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:09.097505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.107790 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fe8d-7acb-4a45-847a-d2cc6d5ae8c8 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.112469 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.113128 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.113410 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.113682 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.114420 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.116039 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.126412 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1405/2804] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:09.130435 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.141031 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2ce44-6637-4b7d-8bd2-9519eaca5f48 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.146333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6ebb-1a8e-4c0a-ad27-f5e93ed3c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.147005 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6ebb-1a8e-4c0a-ad27-f5e93ed3c60c admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.147286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6ebb-1a8e-4c0a-ad27-f5e93ed3c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.147560 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6ebb-1a8e-4c0a-ad27-f5e93ed3c60c admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.148340 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef6ebb-1a8e-4c0a-ad27-f5e93ed3c60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.149979 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef6ebb-1a8e-4c0a-ad27-f5e93ed3c60c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.160882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1400/2805] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:09.165070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.175212 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c373e-50bf-4146-bd7f-ff15dadc9c8c admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.180330 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.181226 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.181491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.181758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] PATH_INFO: `/v3/projects/60003837a07049fc98514c95c49701fc/users/464ae0af95084ddf98f8822a23201b8a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.197694 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=60003837a07049fc98514c95c49701fc, user_id=464ae0af95084ddf98f8822a23201b8a, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.200366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.216350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1406/2806] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:09 2022] PUT /identity/v3/projects/60003837a07049fc98514c95c49701fc/users/464ae0af95084ddf98f8822a23201b8a/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:09.226850 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.227117 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.227430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.255539 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] MFA Rules not processed for user `464ae0af95084ddf98f8822a23201b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:09.256819 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:09.316475 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:09.326500 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1401/2807] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:09.331095 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.358471 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:09.400267 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f8e65-9b6c-42d6-976f-359500b8aeeb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.405137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.405802 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.406080 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.406351 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.407210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.409155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.440479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1407/2808] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:09 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:09.444758 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.456296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71f804-05cb-497b-9a10-c0884774cb9e None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.517308 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.517988 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.518268 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.518543 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.519236 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.521210 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.542802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1402/2809] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:09 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:09.547140 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.557549 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57826c1b-6cb8-4cd0-b141-eb9b7cb39864 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.562154 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.562874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.563156 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.563432 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.564207 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.567316 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.577808 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1408/2810] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:09.583961 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.595296 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a269-df1e-4bf1-82af-490953f76c5d admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.601273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.602200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.602485 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.602746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] PATH_INFO: `/v3/projects/c3d796f8114e424b97fa5e2f1a45106e/users/09bc1ca14e7146f582e02cdec78414e8/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.618973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d796f8114e424b97fa5e2f1a45106e, user_id=09bc1ca14e7146f582e02cdec78414e8, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.621703 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.636957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1403/2811] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:09 2022] PUT /identity/v3/projects/c3d796f8114e424b97fa5e2f1a45106e/users/09bc1ca14e7146f582e02cdec78414e8/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:09.641630 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.652196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df508f44-3858-47a5-b2e8-609c55e84f1a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.711195 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ba07e-31c7-4773-9dee-25a7c8cd1a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.711902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ba07e-31c7-4773-9dee-25a7c8cd1a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.712199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ba07e-31c7-4773-9dee-25a7c8cd1a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.712479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ba07e-31c7-4773-9dee-25a7c8cd1a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.713216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ba07e-31c7-4773-9dee-25a7c8cd1a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.714841 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ba07e-31c7-4773-9dee-25a7c8cd1a2e admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.724615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1409/2812] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:09.728823 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.739398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ebe2e-e2b9-4b8c-9411-f0c9d0d7a223 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.744090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.744982 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.745263 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.745544 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] PATH_INFO: `/v3/projects/c3d796f8114e424b97fa5e2f1a45106e/users/09bc1ca14e7146f582e02cdec78414e8/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.747509 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d796f8114e424b97fa5e2f1a45106e, user_id=09bc1ca14e7146f582e02cdec78414e8, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.750217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.760984 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1404/2813] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:09 2022] PUT /identity/v3/projects/c3d796f8114e424b97fa5e2f1a45106e/users/09bc1ca14e7146f582e02cdec78414e8/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:09.771766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.772092 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.772372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.801008 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] MFA Rules not processed for user `09bc1ca14e7146f582e02cdec78414e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:09.802230 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:09.871980 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1410/2814] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:09.884009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:09.885968 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:09.945882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978476-b5fd-449b-869c-a26b34caa3c2 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:09.949750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:09.950390 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:09.950669 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:09.950938 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:09.960528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:09.961973 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:09.971769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1405/2815] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:10.055305 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:10.066100 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b6f04-ac5b-45a6-bf48-c425bfd685ea None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:10.070381 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc64442-7b47-4c02-8ee1-4496cde4ef28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:10.071031 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc64442-7b47-4c02-8ee1-4496cde4ef28 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:10.071310 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc64442-7b47-4c02-8ee1-4496cde4ef28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:10.071571 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc64442-7b47-4c02-8ee1-4496cde4ef28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:10.088901 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:10.131603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc64442-7b47-4c02-8ee1-4496cde4ef28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:10.133102 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc64442-7b47-4c02-8ee1-4496cde4ef28 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:10.142117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1411/2816] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:10.432790 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:10.486084 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:10.503693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f481-9ae6-4498-a578-d6234e5431cc service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:10.508562 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:10.509495 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:10.509773 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:10.510052 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] PATH_INFO: `/v3/users/464ae0af95084ddf98f8822a23201b8a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:10.511284 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=464ae0af95084ddf98f8822a23201b8a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:10.513242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:10.565746 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464ae0af95084ddf98f8822a23201b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:10.566483 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464ae0af95084ddf98f8822a23201b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:10.576205 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464ae0af95084ddf98f8822a23201b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:10.576932 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464ae0af95084ddf98f8822a23201b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:10.590893 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464ae0af95084ddf98f8822a23201b8a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:10.605081 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1406/2817] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:10 2022] DELETE /identity/v3/users/464ae0af95084ddf98f8822a23201b8a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:10.610397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:10.610677 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:10.610954 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:10.629519 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:10.630750 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:10.696347 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1412/2818] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:10.706357 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:10.707936 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:10.766637 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac829d-1b77-4813-8732-951351c10e56 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:10.770674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:10.771311 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:10.771593 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:10.771889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:10.781156 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:10.782913 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:10.793102 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1407/2819] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:11.081995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:11.083723 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:11.097247 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d6be-33d1-4c34-9fff-fb2d0598732a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:11.103210 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:11.104135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:11.104415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:11.104826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] PATH_INFO: `/v3/projects/60003837a07049fc98514c95c49701fc` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:11.106000 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60003837a07049fc98514c95c49701fc)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:11.107963 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:11.156561 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60003837a07049fc98514c95c49701fc', 'request_id': 'req-6f864656-a16f-42e1-b942-a2498c753710'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.162157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60003837a07049fc98514c95c49701fc', 'request_id': 'req-6f864656-a16f-42e1-b942-a2498c753710'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.172932 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60003837a07049fc98514c95c49701fc', 'request_id': 'req-6f864656-a16f-42e1-b942-a2498c753710'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.184795 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] The token cache is being invalidate because project 60003837a07049fc98514c95c49701fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:11.185083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.185745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.187335 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1413/2820] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:11 2022] DELETE /identity/v3/projects/60003837a07049fc98514c95c49701fc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:11.192780 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:11.204160 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d3e73-cfdc-495b-b116-61649feb5f63 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:11.267537 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:11.269062 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:11.269559 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:11.270051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] PATH_INFO: `/v3/users/09bc1ca14e7146f582e02cdec78414e8` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:11.272259 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09bc1ca14e7146f582e02cdec78414e8)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:11.275172 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:11.292270 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:11.333569 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09bc1ca14e7146f582e02cdec78414e8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.334215 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09bc1ca14e7146f582e02cdec78414e8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.342874 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09bc1ca14e7146f582e02cdec78414e8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.343678 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09bc1ca14e7146f582e02cdec78414e8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.355277 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-18f0af47-ff65-4407-b146-99a0dbbc3743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09bc1ca14e7146f582e02cdec78414e8'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.367478 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1408/2821] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:11 2022] DELETE /identity/v3/users/09bc1ca14e7146f582e02cdec78414e8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:11.654483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:11.656196 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:11.670303 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864656-a16f-42e1-b942-a2498c753710 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:11.716620 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:11.734355 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:11.735386 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:11.735648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:11.735948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] PATH_INFO: `/v3/projects/c3d796f8114e424b97fa5e2f1a45106e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:11.737115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d796f8114e424b97fa5e2f1a45106e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:11.739053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:11.788718 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d796f8114e424b97fa5e2f1a45106e', 'request_id': 'req-1df8db56-871a-4677-afe3-b06385ea0b08'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.794632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d796f8114e424b97fa5e2f1a45106e', 'request_id': 'req-1df8db56-871a-4677-afe3-b06385ea0b08'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.802439 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d796f8114e424b97fa5e2f1a45106e', 'request_id': 'req-1df8db56-871a-4677-afe3-b06385ea0b08'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.813595 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] The token cache is being invalidate because project c3d796f8114e424b97fa5e2f1a45106e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:11.813880 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.814505 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1df8db56-871a-4677-afe3-b06385ea0b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:11.816060 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1414/2822] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:11 2022] DELETE /identity/v3/projects/c3d796f8114e424b97fa5e2f1a45106e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:11.821058 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:11.821348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:11.821625 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:11.840998 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:11.842223 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:11.907760 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1409/2823] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:11.971129 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:11.971598 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:11.972010 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:11.993629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:11.994834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:12.003501 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1415/2824] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:12.007768 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.017958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bebb2-34bf-4c90-a6e0-de5a7ef11eeb None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.022975 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.023591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.023889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.024219 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.024956 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:11:12.025261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.026873 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.038845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1410/2825] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:11:12.043117 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.052615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96df0a0-4079-4ea6-81cc-07269d5a2ba0 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.058990 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.059645 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.059948 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.060218 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.060907 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.062978 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.100933 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1416/2826] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:12 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:12.105381 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.116106 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79255031-52f5-421a-ab8d-9c4132aac819 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.173177 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.173841 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.174126 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.174405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.175095 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.177050 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.198906 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1411/2827] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:12 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:12.203165 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.214121 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bd2c-3725-4477-9199-dac815231cef admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.219902 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.220594 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.220874 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.221143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.221870 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.223462 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.234091 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1417/2828] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:12.238191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.248198 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88873d7-bddf-4144-ba97-ff80882ef962 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.253591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b950d-85b9-4994-bfeb-337be33654d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.254237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b950d-85b9-4994-bfeb-337be33654d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.254535 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b950d-85b9-4994-bfeb-337be33654d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.254805 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b950d-85b9-4994-bfeb-337be33654d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.255526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b950d-85b9-4994-bfeb-337be33654d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.257145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b950d-85b9-4994-bfeb-337be33654d3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.259342 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:12.269965 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1412/2829] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:12.274152 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.284667 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe51501-af50-4e58-8b81-156238cb20b1 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.290038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.290856 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.291125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.291395 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] PATH_INFO: `/v3/projects/b8d82318f396402eadcc7ea26128928c/users/00b99e511592424b865fc93944f4a29d/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.308070 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d82318f396402eadcc7ea26128928c, user_id=00b99e511592424b865fc93944f4a29d, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.310704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.327995 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1418/2830] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:12 2022] PUT /identity/v3/projects/b8d82318f396402eadcc7ea26128928c/users/00b99e511592424b865fc93944f4a29d/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:12.339286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.339610 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.339966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.368310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] MFA Rules not processed for user `00b99e511592424b865fc93944f4a29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:12.369800 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:12.439464 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1413/2831] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:12.444472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.507470 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0d72-fa0d-4645-b8f9-b89521f41472 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.512960 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.513610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.513892 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.514163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.514840 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.517007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.529719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:12.547516 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1419/2832] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:12 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:12.551723 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.562957 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c00937-ea9f-423d-b5e3-87027af5d952 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.641019 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.641700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.642090 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.642434 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.643345 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.645313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.668764 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1414/2833] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:12 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:11:12.674106 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.685281 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96352-820f-4bb0-af6d-8fbc0fb09ad1 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.691067 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.691766 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.692083 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.692372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.693086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.694705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.705587 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1420/2834] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:12.710097 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.720620 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739ad7d-2194-4660-bd1b-fc1405a8e95b admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.725798 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.726687 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.726963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.727237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] PATH_INFO: `/v3/projects/29651838f4a243b8b3645a15cd313e6b/users/3a05f38b60d74e789c1404da338421fa/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.742479 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=29651838f4a243b8b3645a15cd313e6b, user_id=3a05f38b60d74e789c1404da338421fa, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.745208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.760408 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1415/2835] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:12 2022] PUT /identity/v3/projects/29651838f4a243b8b3645a15cd313e6b/users/3a05f38b60d74e789c1404da338421fa/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:12.764552 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.774900 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd2a4c-aca2-45c8-859f-37485c1809ac admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.830348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186b147-3beb-4572-a25b-773e23165100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.830936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186b147-3beb-4572-a25b-773e23165100 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.831216 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186b147-3beb-4572-a25b-773e23165100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.831489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186b147-3beb-4572-a25b-773e23165100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.832248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186b147-3beb-4572-a25b-773e23165100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.833846 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186b147-3beb-4572-a25b-773e23165100 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.843445 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1421/2836] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:12.847787 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:12.859086 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f84ed-5bb8-4201-b889-940e71ecbbbd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:12.864801 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:12.865685 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.865963 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.866218 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] PATH_INFO: `/v3/projects/29651838f4a243b8b3645a15cd313e6b/users/3a05f38b60d74e789c1404da338421fa/roles/d5c3d840a1cc493686f45bd92d196eac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.868391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29651838f4a243b8b3645a15cd313e6b, user_id=3a05f38b60d74e789c1404da338421fa, role_id=d5c3d840a1cc493686f45bd92d196eac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:12.871002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:12.882608 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1416/2837] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:12 2022] PUT /identity/v3/projects/29651838f4a243b8b3645a15cd313e6b/users/3a05f38b60d74e789c1404da338421fa/roles/d5c3d840a1cc493686f45bd92d196eac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:12.893274 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:12.893565 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:12.893829 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:12.920666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] MFA Rules not processed for user `3a05f38b60d74e789c1404da338421fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:12.921927 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:12.991762 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1422/2838] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:11:13.004527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:13.006346 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:13.050237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:13.066824 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2dffc-7daa-4e5b-82a8-e87016a384d6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:13.071026 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:13.071672 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:13.071980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:13.072258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:13.132242 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:13.133697 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:13.142912 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1417/2839] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:17.454720 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:17.456444 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:17.473704 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59400d-6a36-4a33-9e21-3a383e45bd02 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:17.478231 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ff43f-2337-4618-8677-03c4f01f09b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:17.478872 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff43f-2337-4618-8677-03c4f01f09b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:17.479145 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff43f-2337-4618-8677-03c4f01f09b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:17.479414 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff43f-2337-4618-8677-03c4f01f09b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:17.490651 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ff43f-2337-4618-8677-03c4f01f09b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:17.492169 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ff43f-2337-4618-8677-03c4f01f09b7 service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:17.502098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1423/2840] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:22.433430 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:22.501775 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc370534-0fbe-444a-b5b7-d0f1dbaf149d service cinder] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:22.507108 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:22.507999 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:22.508282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:22.508554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] PATH_INFO: `/v3/users/00b99e511592424b865fc93944f4a29d` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:22.509786 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b99e511592424b865fc93944f4a29d)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:22.511708 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:22.565335 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b99e511592424b865fc93944f4a29d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:22.566029 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b99e511592424b865fc93944f4a29d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:22.574700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b99e511592424b865fc93944f4a29d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:22.575313 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b99e511592424b865fc93944f4a29d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:22.587235 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b99e511592424b865fc93944f4a29d'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:22.600073 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1418/2841] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:22 2022] DELETE /identity/v3/users/00b99e511592424b865fc93944f4a29d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:22.604876 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:22.605399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:22.605502 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:22.623372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:22.624616 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:22.669127 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:22.685477 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1424/2842] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:11:22.696834 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:22.698431 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:22.713632 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:22.763738 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35afdc-2e7c-4016-9861-885572c20cd1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:22.768299 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:22.768958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:22.769239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:22.769700 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:22.779674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:22.781237 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:22.791650 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1419/2843] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:11:23.073586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:23.075013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:23.087332 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84704dbc-f06a-4f19-8b4b-40127b493a6b None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:23.092262 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:23.093125 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.093399 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.093666 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] PATH_INFO: `/v3/projects/b8d82318f396402eadcc7ea26128928c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.094871 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d82318f396402eadcc7ea26128928c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:23.097016 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:23.146452 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d82318f396402eadcc7ea26128928c', 'request_id': 'req-a2785ae9-4595-4d81-96c5-2f48edd482c0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.151754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d82318f396402eadcc7ea26128928c', 'request_id': 'req-a2785ae9-4595-4d81-96c5-2f48edd482c0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.160378 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d82318f396402eadcc7ea26128928c', 'request_id': 'req-a2785ae9-4595-4d81-96c5-2f48edd482c0'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.171757 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] The token cache is being invalidate because project b8d82318f396402eadcc7ea26128928c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:23.172045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.172693 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.177205 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1425/2844] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:23 2022] DELETE /identity/v3/projects/b8d82318f396402eadcc7ea26128928c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:23.178258 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:23.189379 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9fe78-a9b6-4780-bef5-8439cf9f98ff service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:23.249462 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:23.250310 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.250627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.250889 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] PATH_INFO: `/v3/users/3a05f38b60d74e789c1404da338421fa` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.252244 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a05f38b60d74e789c1404da338421fa)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:23.254199 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:23.307831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a05f38b60d74e789c1404da338421fa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.308527 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a05f38b60d74e789c1404da338421fa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.317661 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a05f38b60d74e789c1404da338421fa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.318300 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a05f38b60d74e789c1404da338421fa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.331145 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-f815ee86-7f03-4202-9051-97e0524e6b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a05f38b60d74e789c1404da338421fa'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.343309 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1420/2845] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:23 2022] DELETE /identity/v3/users/3a05f38b60d74e789c1404da338421fa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:11:23.632020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:23.633758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:23.647415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2785ae9-4595-4d81-96c5-2f48edd482c0 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:23.707509 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:23.708472 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.708728 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.709006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] PATH_INFO: `/v3/projects/29651838f4a243b8b3645a15cd313e6b` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.710173 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29651838f4a243b8b3645a15cd313e6b)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:23.712115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:23.764415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29651838f4a243b8b3645a15cd313e6b', 'request_id': 'req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.769691 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29651838f4a243b8b3645a15cd313e6b', 'request_id': 'req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.779546 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29651838f4a243b8b3645a15cd313e6b', 'request_id': 'req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.790881 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] The token cache is being invalidate because project 29651838f4a243b8b3645a15cd313e6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:23.791167 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.791889 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-26723dd7-7ef7-4c48-8f3a-7b51ac7457d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:23.793531 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1426/2846] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:23 2022] DELETE /identity/v3/projects/29651838f4a243b8b3645a15cd313e6b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:23.799631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.800128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.800257 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.820653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:23.821845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:23.852640 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:23.883627 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1421/2847] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:23.896379 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.896747 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.897046 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.907748 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:23.921648 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:23.923265 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:23.931899 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1427/2848] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:23.936398 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:23.946489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f812b5-abdf-471d-87cb-49b28074dfd1 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:23.951832 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:23.952526 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.952806 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.953076 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.953817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:11:23.954120 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:23.955722 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:23.968088 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1422/2849] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:11:23.972466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:23.981895 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585df4cf-3713-4951-8b39-ce251ccbc8c4 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:23.987040 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:23.987683 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:23.987986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:23.988382 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:23.989051 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:23.990941 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:24.017431 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1428/2850] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:23 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:24.021744 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:24.032496 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f0e4d-752b-479a-a73a-620430f57115 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:24.093471 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:24.094176 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:24.094457 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:24.094763 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:24.095445 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:24.097436 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:24.121382 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1423/2851] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:24 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:11:24.127743 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:24.138862 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6e4d1-b9c8-440d-82c1-bc28d7a7ecfb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:24.144183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:24.145050 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:24.145329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:24.145602 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:24.146329 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:24.147953 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:24.158716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1429/2852] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:24.163769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:24.174486 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92ef3-b30a-4f64-ac25-7478109a34d6 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:24.180533 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100dd3ae-4c84-44e2-8af0-5a6b4dd789f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:24.181191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100dd3ae-4c84-44e2-8af0-5a6b4dd789f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:24.181474 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100dd3ae-4c84-44e2-8af0-5a6b4dd789f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:24.181751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100dd3ae-4c84-44e2-8af0-5a6b4dd789f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:24.182491 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100dd3ae-4c84-44e2-8af0-5a6b4dd789f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:24.184122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100dd3ae-4c84-44e2-8af0-5a6b4dd789f3 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:24.194914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1424/2853] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:24.199174 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:24.209654 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e60ef4-0e2f-4851-8629-b7ba566c49d5 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:24.214784 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:24.215668 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:24.216038 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:24.216317 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] PATH_INFO: `/v3/projects/e0794b714cc946ba8fe58f95c6533464/users/76d1f4ca06e94b5b9942d94a079eede6/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:24.231559 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0794b714cc946ba8fe58f95c6533464, user_id=76d1f4ca06e94b5b9942d94a079eede6, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:24.234180 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:24.248455 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1430/2854] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:24 2022] PUT /identity/v3/projects/e0794b714cc946ba8fe58f95c6533464/users/76d1f4ca06e94b5b9942d94a079eede6/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:11:24.259648 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:24.259971 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:24.260282 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:24.289100 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] MFA Rules not processed for user `76d1f4ca06e94b5b9942d94a079eede6`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:24.290291 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:24.354802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1425/2855] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:24.368834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:24.370558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:24.431508 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5776c-4382-4bb0-8cb1-8a86eb69b01d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:24.435921 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc953a-9c46-491c-965c-660ad2ade5e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a98b63a4ad14d7399938ffb9c6b018a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:24.436614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc953a-9c46-491c-965c-660ad2ade5e3 service glance] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:24.436894 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc953a-9c46-491c-965c-660ad2ade5e3 service glance] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:24.437142 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc953a-9c46-491c-965c-660ad2ade5e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:24.446454 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc953a-9c46-491c-965c-660ad2ade5e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:24.447919 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc953a-9c46-491c-965c-660ad2ade5e3 service glance] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:24.457433 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1431/2856] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:34.352078 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:34.452803 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:41.117191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:41.187082 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49694b21-b069-4bd0-88f3-dbd3ab144106 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:41.192083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:41.193009 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:41.193273 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:41.193576 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] PATH_INFO: `/v3/users/76d1f4ca06e94b5b9942d94a079eede6` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:41.194802 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d1f4ca06e94b5b9942d94a079eede6)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:41.197043 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:41.251060 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d1f4ca06e94b5b9942d94a079eede6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.251762 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d1f4ca06e94b5b9942d94a079eede6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.261312 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d1f4ca06e94b5b9942d94a079eede6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.262008 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d1f4ca06e94b5b9942d94a079eede6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.273958 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d1f4ca06e94b5b9942d94a079eede6'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.286749 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1426/2857] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:11:41 2022] DELETE /identity/v3/users/76d1f4ca06e94b5b9942d94a079eede6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:41.291759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:41.292368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:41.292368 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:41.310163 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:41.311358 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:41.373969 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1432/2858] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:41.383952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:41.385429 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:41.444333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8f095-e7bd-4c9e-b406-c5ddf79dfcec admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:41.448440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82269293-8b66-4bf1-a2fd-d4ebccea9b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:41.449077 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269293-8b66-4bf1-a2fd-d4ebccea9b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:41.449351 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269293-8b66-4bf1-a2fd-d4ebccea9b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:41.449614 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269293-8b66-4bf1-a2fd-d4ebccea9b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:41.459045 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82269293-8b66-4bf1-a2fd-d4ebccea9b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:41.460508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82269293-8b66-4bf1-a2fd-d4ebccea9b04 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:41.469769 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1427/2859] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:41.759544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:41.761151 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:41.776135 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ca86e-f9a1-469f-b505-d6ee1a28ba02 None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:41.782798 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:41.783679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:41.783972 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:41.784248 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] PATH_INFO: `/v3/projects/e0794b714cc946ba8fe58f95c6533464` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:41.785548 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0794b714cc946ba8fe58f95c6533464)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:41.787534 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:41.837745 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0794b714cc946ba8fe58f95c6533464', 'request_id': 'req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.843007 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0794b714cc946ba8fe58f95c6533464', 'request_id': 'req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.851987 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0794b714cc946ba8fe58f95c6533464', 'request_id': 'req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.864615 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] The token cache is being invalidate because project e0794b714cc946ba8fe58f95c6533464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:11:41.864977 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.865774 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-a98e4ba2-0208-48d4-8c07-f519e5f8e6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:11:41.867767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1433/2860] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:11:41 2022] DELETE /identity/v3/projects/e0794b714cc946ba8fe58f95c6533464 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:41.873030 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:41.873348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:41.873624 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:41.892227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:41.893336 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:41.910797 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:41.958083 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1428/2861] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:41.966936 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:41.967222 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:41.967491 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:41.992098 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:41.993279 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:42.002415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1434/2862] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:42.007283 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.020862 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b9ad4-061f-44af-9dc1-b6c53d10dba5 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.026261 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.026911 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.027191 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.027463 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] PATH_INFO: `/v3/domains` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.028207 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80150) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:11:42.028521 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.030142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.042791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1429/2863] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 10:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:11:42.047214 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.057110 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68e9b3-ccca-4d42-80dc-7d543a90aa3c None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.062717 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.063366 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.063640 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.063909 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.064638 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.066591 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.094833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1435/2864] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 10:11:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:42.100423 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.111180 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4fca3-fe67-437f-8ed8-4ff6f0138399 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.177634 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.178651 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.178900 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.179166 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] PATH_INFO: `/v3/users` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.179851 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.181808 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.204926 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1430/2865] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 10:11:42 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:11:42.209219 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.219675 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27261bb3-765b-467e-947d-5ec235895e07 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.224818 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.225479 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.225758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.226027 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.226749 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.228373 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.239250 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1436/2866] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:11:42.243362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.253232 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4ee1-0a52-4a0b-8021-0a3bc6c4c7bd admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.258577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39571987-d12e-44ef-ae68-dc8cea3b5958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.259239 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39571987-d12e-44ef-ae68-dc8cea3b5958 admin admin] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.259528 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39571987-d12e-44ef-ae68-dc8cea3b5958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.259799 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39571987-d12e-44ef-ae68-dc8cea3b5958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.260591 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39571987-d12e-44ef-ae68-dc8cea3b5958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.262168 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39571987-d12e-44ef-ae68-dc8cea3b5958 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.273262 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1431/2867] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 10:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:11:42.277323 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.288037 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e321ef-afce-4652-b13a-7cec110f5b4a admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.292891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.293785 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.294064 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.294337 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] PATH_INFO: `/v3/projects/12b12537f14f4f56b0b2b5b65ca459a7/users/44e759dcbb0a4b57b13449ed7b195b2e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.310136 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b12537f14f4f56b0b2b5b65ca459a7, user_id=44e759dcbb0a4b57b13449ed7b195b2e, role_id=b16e6fc6c0ee491e8078bc37c8c12d0c)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.312835 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.328104 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1437/2868] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 10:11:42 2022] PUT /identity/v3/projects/12b12537f14f4f56b0b2b5b65ca459a7/users/44e759dcbb0a4b57b13449ed7b195b2e/roles/b16e6fc6c0ee491e8078bc37c8c12d0c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:11:42.339341 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b8854-a556-40d9-b647-0f8dabe5128f None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.339721 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b8854-a556-40d9-b647-0f8dabe5128f None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.340087 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b8854-a556-40d9-b647-0f8dabe5128f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.370367 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-be0b8854-a556-40d9-b647-0f8dabe5128f None None] MFA Rules not processed for user `44e759dcbb0a4b57b13449ed7b195b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:42.371586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-be0b8854-a556-40d9-b647-0f8dabe5128f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:42.454908 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1432/2869] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:42.468689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:42.470754 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:42.494629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:42.534459 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc144a-542e-4c4e-9347-31f17fd46025 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:42.538456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15924f58d53145ae83b3b4cfbbb02ef7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:42.539103 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:42.539385 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:42.539658 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:42.548202 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:42.549635 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:42.559456 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1438/2870] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:43.948791 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:43.949472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:43.949472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:43.968845 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] MFA Rules not processed for user `44e759dcbb0a4b57b13449ed7b195b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:43.970051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:43.978128 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1433/2871] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:11:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:44.470986 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:44.472496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:44.545417 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554175dc-97c1-4881-86c3-c3625633beef service cinder] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:44.551124 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:44.551799 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:44.552115 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:44.552384 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:44.554742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:44.566006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:44.567767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:44.578252 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1439/2872] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:47.123362 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:47.126190 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:47.145333 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:47.226016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682d634-654b-4b26-8eb0-4e2a4e5e78cc None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:47.232507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:47.233489 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:47.233860 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:47.234222 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:47.237360 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:47.255966 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:47.257696 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0919b8-824b-4ca6-abea-bcf3d0de1cbc service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:47.269649 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1434/2873] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:52.705593 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:52.707101 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:52.723350 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9320e-68bd-4672-83e3-11972c320159 service placement] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:52.729496 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:52.730206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:52.730542 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:52.730826 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:52.732834 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:52.751610 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:52.753483 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:52.767114 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1440/2874] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:11:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:11:57.264750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=29 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:57.819584 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3578ed-dc5c-4882-bb03-13c257c8bdeb None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:57.819924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3578ed-dc5c-4882-bb03-13c257c8bdeb None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:57.820347 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3578ed-dc5c-4882-bb03-13c257c8bdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:57.842983 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-eb3578ed-dc5c-4882-bb03-13c257c8bdeb None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:57.844227 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-eb3578ed-dc5c-4882-bb03-13c257c8bdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:57.904707 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1435/2875] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:11:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:57.963132 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:57.964636 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:58.035611 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667a07-c244-4c4b-914d-1503de9d4bd7 service nova] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:58.041189 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:58.041850 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:58.042137 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:58.042415 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:58.052108 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:58.054107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:58.062842 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1441/2876] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:11:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:11:58.596161 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c04d-7ff1-411c-b162-a92c68f89b35 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:58.596628 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c04d-7ff1-411c-b162-a92c68f89b35 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:58.596741 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c04d-7ff1-411c-b162-a92c68f89b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:58.616647 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-78f4c04d-7ff1-411c-b162-a92c68f89b35 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:58.617844 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-78f4c04d-7ff1-411c-b162-a92c68f89b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:58.625914 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1436/2877] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 10:11:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:58.642338 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:58.654544 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ba800-a197-43ce-ae7c-584f4073dec2 service swift] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:58.660553 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:58.661206 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:58.661490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:58.661763 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:58.669970 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:58.671557 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:58.680020 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1442/2878] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:11:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:11:58.891449 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824ad1-18ff-4e52-88ee-8e3c56a6c9e2 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:58.891733 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824ad1-18ff-4e52-88ee-8e3c56a6c9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:58.892051 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03824ad1-18ff-4e52-88ee-8e3c56a6c9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:58.910623 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-03824ad1-18ff-4e52-88ee-8e3c56a6c9e2 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:58.912096 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-03824ad1-18ff-4e52-88ee-8e3c56a6c9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:58.923107 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1437/2879] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 10:11:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:58.938143 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:58.948999 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604173f0-deb5-4610-a6f7-fe5265dba437 service swift] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:58.954716 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:58.955348 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:58.955625 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:58.955945 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:58.965490 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:58.967558 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:58.978574 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1443/2880] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:11:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 10:11:59.322141 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] REQUEST_METHOD: `POST` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:59.322421 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:59.322699 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:59.341492 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.auth.core [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] MFA Rules not processed for user `cc7643dc9f834584973cb7fdedbb4d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80150) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:11:59.342727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:11:59.350002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1438/2881] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 10:11:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:11:59.362679 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:11:59.368073 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:11:59.373776 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393403b-ec0d-4122-bc20-587306b7c541 service swift] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:11:59.379788 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be9a2-96a4-468d-9c3e-8e39417b73cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64bf7840b5fd4b3b9dd2b8e18b4495b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:11:59.380674 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be9a2-96a4-468d-9c3e-8e39417b73cb service swift] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:11:59.381017 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be9a2-96a4-468d-9c3e-8e39417b73cb service swift] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:11:59.381357 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be9a2-96a4-468d-9c3e-8e39417b73cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:11:59.389939 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be9a2-96a4-468d-9c3e-8e39417b73cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:11:59.391624 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be9a2-96a4-468d-9c3e-8e39417b73cb service swift] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:11:59.400293 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1444/2882] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 10:11:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:12:01.558508 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:01.626674 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed02bc-cce5-4b9f-9195-845825cfa5b0 None None] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:01.632033 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:01.632924 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:01.633315 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:01.634047 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] PATH_INFO: `/v3/users/44e759dcbb0a4b57b13449ed7b195b2e` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:01.635158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e759dcbb0a4b57b13449ed7b195b2e)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:01.637428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:01.702477 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e759dcbb0a4b57b13449ed7b195b2e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:01.703280 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e759dcbb0a4b57b13449ed7b195b2e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:01.714724 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e759dcbb0a4b57b13449ed7b195b2e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:01.715693 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e759dcbb0a4b57b13449ed7b195b2e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:01.729618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e759dcbb0a4b57b13449ed7b195b2e'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:01.742203 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1439/2883] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:12:01 2022] DELETE /identity/v3/users/44e759dcbb0a4b57b13449ed7b195b2e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:12:01.746712 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] REQUEST_METHOD: `POST` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:01.747204 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:01.747375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:01.766425 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.auth.core [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] MFA Rules not processed for user `7c92e7defa704085b3b2760be29aa346`. Rule list: `[]` (Enabled: `True`). {{(pid=80149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:12:01.767701 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:01.836045 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1445/2884] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:12:01.846513 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:01.848248 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:01.855598 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:12:01.916142 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73cabd-75d6-4c3b-b1c1-568d1b178718 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:01.920502 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:01.921159 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:01.921440 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:01.921716 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:01.931980 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:01.933437 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:01.942831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1440/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:12:02.118540 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:02.120066 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:02.134307 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6159183-e39f-4af9-8f6f-2d78aacb7d4a None None] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:02.140387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:02.141464 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:02.141741 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:02.142162 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] PATH_INFO: `/v3/projects/12b12537f14f4f56b0b2b5b65ca459a7` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:02.143957 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b12537f14f4f56b0b2b5b65ca459a7)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:02.146551 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:02.196489 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b12537f14f4f56b0b2b5b65ca459a7', 'request_id': 'req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:02.203032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b12537f14f4f56b0b2b5b65ca459a7', 'request_id': 'req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:02.212586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b12537f14f4f56b0b2b5b65ca459a7', 'request_id': 'req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:02.224935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] The token cache is being invalidate because project 12b12537f14f4f56b0b2b5b65ca459a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:12:02.225249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:02.225922 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:02.227738 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1446/2886] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:12:02 2022] DELETE /identity/v3/projects/12b12537f14f4f56b0b2b5b65ca459a7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:12:04.299577 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:04.309619 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aad0ad-e231-4227-a2b6-15dccf5975f6 service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:04.364391 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:04.365070 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:04.365348 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:04.365618 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:04.367549 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:04.377505 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:04.378952 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:04.387428 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1441/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:12:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:12:04.490387 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:04.492364 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:04.507742 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d7d5f-448a-4b3b-a4d4-b296a22e32ab admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:04.512530 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:04.513249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:04.513463 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:04.513729 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:04.515758 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:04.603006 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:04.604497 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:04.609466 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:12:04.618227 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1447/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:12:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:12:14.383484 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=50 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:12:14.621670 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:12:39.425564 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:39.428146 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:39.445493 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e73d05-02b2-4b48-8f3f-e424e87629ab service neutron] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:39.451498 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14dd1de1fe914f319017818e571732de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:39.451752 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:39.452054 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:39.452329 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:39.454123 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:39.466909 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:39.468536 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:39.491208 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1442/2889] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:12:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:12:45.093026 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:45.102603 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd9a0e-4e65-4d4e-a47e-3a85b8c9e7c2 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:45.106711 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b086db83a6b47fb98d450bb7e1c8026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:45.107356 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:45.107632 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:45.107934 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:45.109892 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:45.173997 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:45.175467 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:45.187538 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1448/2890] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:12:48.353371 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:48.355136 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:48.452470 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c234eb-3555-4b47-84cd-cd2b950a7520 service nova] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:48.457817 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:48.458949 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:48.459552 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:48.460080 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:48.464870 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:48.484964 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:48.487000 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:48.505774 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1443/2891] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:12:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:12:51.423705 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:51.425249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:51.439923 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dc2e4-f1f7-4694-af44-7ef77877d773 service neutron] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:51.445288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:51.446175 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:51.446480 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:51.446752 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] PATH_INFO: `/v3/users/ead8ee09f8514e75882c877042764a92` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:51.448107 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead8ee09f8514e75882c877042764a92)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:51.450032 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:51.501644 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead8ee09f8514e75882c877042764a92'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.502283 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead8ee09f8514e75882c877042764a92'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.502861 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead8ee09f8514e75882c877042764a92'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.511191 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead8ee09f8514e75882c877042764a92'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.525576 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead8ee09f8514e75882c877042764a92'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.538288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1449/2892] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:12:51 2022] DELETE /identity/v3/users/ead8ee09f8514e75882c877042764a92 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:12:51.721831 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:51.733653 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560872d-556c-434d-9a33-2c5094f3b206 service placement] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:51.799507 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:51.800458 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:51.801091 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:51.801411 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] PATH_INFO: `/v3/projects/108ee256fba04d1b83fe00c166742f0b` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:51.802727 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108ee256fba04d1b83fe00c166742f0b)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:51.804680 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:51.858551 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108ee256fba04d1b83fe00c166742f0b', 'request_id': 'req-6c2258db-d544-4457-8f99-19f82aaa8507'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.864154 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108ee256fba04d1b83fe00c166742f0b', 'request_id': 'req-6c2258db-d544-4457-8f99-19f82aaa8507'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.870751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108ee256fba04d1b83fe00c166742f0b', 'request_id': 'req-6c2258db-d544-4457-8f99-19f82aaa8507'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.884737 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] The token cache is being invalidate because project 108ee256fba04d1b83fe00c166742f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:12:51.885104 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.885756 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:51.887550 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1444/2893] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:12:51 2022] DELETE /identity/v3/projects/108ee256fba04d1b83fe00c166742f0b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:12:51.891494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:51.893586 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:51.906942 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d144-314d-423e-871f-f00194b5c740 admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:51.973199 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:51.974096 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:51.974372 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:51.974642 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] PATH_INFO: `/v3/users/504258e47adc476d92eb3db56e71126d` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:51.975833 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=504258e47adc476d92eb3db56e71126d)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:51.977786 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:51.984288 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:12:52.030554 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504258e47adc476d92eb3db56e71126d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.031220 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504258e47adc476d92eb3db56e71126d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.031891 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504258e47adc476d92eb3db56e71126d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.040298 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504258e47adc476d92eb3db56e71126d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.052949 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504258e47adc476d92eb3db56e71126d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.065690 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1450/2894] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:12:51 2022] DELETE /identity/v3/users/504258e47adc476d92eb3db56e71126d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:12:52.248204 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:52.249750 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:52.266826 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2258db-d544-4457-8f99-19f82aaa8507 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:52.271523 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:12:52.333138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:52.334014 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:52.334302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:52.334573 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] PATH_INFO: `/v3/projects/f6bdfc275275414fa807b014ea3360cc` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:52.335935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6bdfc275275414fa807b014ea3360cc)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:52.337977 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:52.390574 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6bdfc275275414fa807b014ea3360cc', 'request_id': 'req-019b2179-3250-4849-af0f-38ca1b78bc4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.396554 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6bdfc275275414fa807b014ea3360cc', 'request_id': 'req-019b2179-3250-4849-af0f-38ca1b78bc4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.403217 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6bdfc275275414fa807b014ea3360cc', 'request_id': 'req-019b2179-3250-4849-af0f-38ca1b78bc4c'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.421855 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] The token cache is being invalidate because project f6bdfc275275414fa807b014ea3360cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80150) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:12:52.422213 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.423321 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:12:52.426133 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1445/2895] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:12:52 2022] DELETE /identity/v3/projects/f6bdfc275275414fa807b014ea3360cc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:02.062183 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:02.417745 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.921614 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.923241 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:38.099500 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eec46-567c-4aae-bee6-586c240058bb admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:38.166055 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:38.166767 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] REQUEST_METHOD: `GET` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:38.167058 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:38.167333 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:38.169158 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:38.181411 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:38.183079 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:38.194155 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1451/2896] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:13:39.415419 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:39.417293 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:39.543330 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b2179-3250-4849-af0f-38ca1b78bc4c admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:39.548243 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:39.549138 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.549734 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.550012 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] PATH_INFO: `/v3/users/fbc71c372e584ddf91290eefb53a973a` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.551200 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc71c372e584ddf91290eefb53a973a)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:39.553158 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:39.602016 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc71c372e584ddf91290eefb53a973a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.602725 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc71c372e584ddf91290eefb53a973a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.611472 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc71c372e584ddf91290eefb53a973a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.612139 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc71c372e584ddf91290eefb53a973a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.623886 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc71c372e584ddf91290eefb53a973a'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.635915 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1446/2897] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:13:39 2022] DELETE /identity/v3/users/fbc71c372e584ddf91290eefb53a973a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:39.810958 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:39.822272 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25f23-2123-4ad6-9932-9f094821b476 service placement] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:39.849585 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=100 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:39.886887 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:39.887771 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.888086 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.888362 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] PATH_INFO: `/v3/projects/84c9f15ea7b84f54a6cfa004bd02de6e` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.889719 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c9f15ea7b84f54a6cfa004bd02de6e)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:39.891629 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:39.938157 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c9f15ea7b84f54a6cfa004bd02de6e', 'request_id': 'req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.943522 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c9f15ea7b84f54a6cfa004bd02de6e', 'request_id': 'req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.952215 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c9f15ea7b84f54a6cfa004bd02de6e', 'request_id': 'req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.963139 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] The token cache is being invalidate because project 84c9f15ea7b84f54a6cfa004bd02de6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:13:39.963397 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.964081 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:39.965673 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1452/2898] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:13:39 2022] DELETE /identity/v3/projects/84c9f15ea7b84f54a6cfa004bd02de6e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:39.970010 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:39.971711 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:39.985676 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989cc5-4ef8-49cf-9760-c1a08361255f admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.009690 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.055659 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.056473 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.056751 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.057021 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] PATH_INFO: `/v3/users/262491bdaffd47adaa2d174b5160e0ac` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.058377 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=262491bdaffd47adaa2d174b5160e0ac)` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.060373 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.110301 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262491bdaffd47adaa2d174b5160e0ac'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.110935 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262491bdaffd47adaa2d174b5160e0ac'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.119962 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262491bdaffd47adaa2d174b5160e0ac'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.120586 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262491bdaffd47adaa2d174b5160e0ac'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.132103 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.notifications [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262491bdaffd47adaa2d174b5160e0ac'} {{(pid=80150) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.144363 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1447/2899] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 10:13:39 2022] DELETE /identity/v3/users/262491bdaffd47adaa2d174b5160e0ac => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:40.329494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Authenticating user token {{(pid=80149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.330935 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.fernet_utils [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:40.344567 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2745216-0fc7-4ea5-91b7-2d5cda9b560d admin admin] Validating token access rules against request {{(pid=80149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.404053 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c92e7defa704085b3b2760be29aa346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e04fd91b7cf34c948c016c3c7a5bd4df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.405217 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.405499 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.405759 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] PATH_INFO: `/v3/projects/bfa9157dcdfd4183a53edc3cf25566e7` {{(pid=80149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.406944 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa9157dcdfd4183a53edc3cf25566e7)` {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.408911 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] RBAC: Authorization granted {{(pid=80149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.455994 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa9157dcdfd4183a53edc3cf25566e7', 'request_id': 'req-ea8e228f-dd55-4f1e-bd51-6f787e399dde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.460689 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.462013 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa9157dcdfd4183a53edc3cf25566e7', 'request_id': 'req-ea8e228f-dd55-4f1e-bd51-6f787e399dde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.470349 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa9157dcdfd4183a53edc3cf25566e7', 'request_id': 'req-ea8e228f-dd55-4f1e-bd51-6f787e399dde'} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.482375 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] The token cache is being invalidate because project bfa9157dcdfd4183a53edc3cf25566e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80149) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:13:40.482775 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.483607 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG keystone.notifications [None req-ea8e228f-dd55-4f1e-bd51-6f787e399dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80149) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:13:40.485249 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: [pid: 80149|app: 0|req: 1453/2900] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 10:13:40 2022] DELETE /identity/v3/projects/bfa9157dcdfd4183a53edc3cf25566e7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:50.141002 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:50.480494 dv-cinder-843309-3-iscsi devstack@keystone.service[80149]: DEBUG dbcounter [-] [80149] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:42.958631 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Authenticating user token {{(pid=80150) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:42.960302 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:18:43.041197 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c967f-5592-4f19-87f8-422d8f492ed1 admin admin] Validating token access rules against request {{(pid=80150) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:43.112882 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8641d4ad6c8a45f5b826bb43a78ee3da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e3088f8a4154610998c2ab87ea883db', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80150) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:43.113534 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] REQUEST_METHOD: `GET` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:43.113821 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:43.114098 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80150) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:43.116122 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.fernet_utils [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80150) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:18:43.127783 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:43.129405 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098101f-c1ae-4d1e-8bb1-b579035373d7 service placement] RBAC: Authorization granted {{(pid=80150) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:43.139286 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: [pid: 80150|app: 0|req: 1448/2901] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 10:18:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:18:53.135266 dv-cinder-843309-3-iscsi devstack@keystone.service[80150]: DEBUG dbcounter [-] [80150] Writing DB stats keystone:SELECT=58 {{(pid=80150) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}